Multimedia in Organisations

1. Multimedia in Organisations BUSS 213 Lecture 6 Creating New Media Media under the Influence of Computation

2. Notices (1)General • Make sure you have downloaded a copy of the BUSS213 Assignment 2- Multimedia Presentation for a Concept, Service, Product or Place • BUSS213websitewhere you can find out the latest Notices and get Lecture Notes, Tutorial Sheets, Assignments etc is: www.uow.edu.au/~rclarke/buss213/buss213.htm • this week tutorial sheet is available on this site for use in the laboratory (open up WORD or simply print it out)

3. Notices (2)Marking Assignment 1 • Assignment 1 is still being marked and will likely be available next lecture… • …however, if I complete these earlier then I will announce this under the Noticessection of the BUSS213 Intranet • your marks will be available either on the notice board on my office door or on the Departments Notice Board adjacent to my office- marks will not be made available on the Intranet

4. Notices (3)Download Assignment 2 • Assignment 2 is due Week 9 (21/9): • is a Director Version 5 assignment- any assignment completed using other versions or multimedia systems will not be marked • you are required to inform me prior to the Lecture Week 7 by email of your selected topic- when composing the email to me please use exactly the following subject name of A2-213-01 • you submit the assignment using the Microcomputer Laboratories Submit system- details will be provided nearer to the completion data and time • there are three compulsory options that each assignment must have- (1) splash screen, (2) credits screen, and an (3) exit option

5. Agenda (1) • in the previous three lectures I have provided information on three of the most important groups of traditional media used in multimedia systems • Why bother? Because: • in order to create good media you need to understand the processesby which media is gathered, manipulated and used- this is why we have concentrated on conventional techniques like framing in film, traditional cell animation and so on, • you cannot create good multimedia unless you understand thecharacteristics and potentials of media types you deal with, and • only when you understand individual media can you appreciate what adding computation can do to create a functional multimedia system

6. Agenda (2) • in the first lecture we said that the major reason new forms of media can be created is that traditional digital media can be transformed from data to processes • we will see that ‘new’ media (or digital media) involves adding computation to existing forms of mediaby adding: • Selection- on event do this • Repetition- repeat this until that • to a specific State for static media or Sequencefor Time-ordered media

7. Agenda (3)Transforming Traditional into New Media • transforming traditional media into new digital media can occur to even the most basic of media types- for example the use of computation to transform text into dimensional typography • we will illustrate that sequence, selection and repetition can work to create entirely new kinds of media (we consider Time Slice Imagery and QuickTime Virtual Reality)

8. Agenda (4)Creating New Media • there are many forms of new digital media which are currently being created- many simply await an application for which they are suited • this was in fact the case for Time Slice Imaging when it was first described in a Scientific American magazine • if we can understand and define media we may be able to create entirely new kinds of passive and interactive experience for users! • we can demonstrate how this is possible by proposing a variation on Time Slice Imagery which we will call Space Time Imaging (STI)

9. Dimensional Typography

10. Dimensional Typography • typography is defined as the art of designing letters while dimensional typography adds a spatial and temporal aspect to traditional flat and static letters • developments in graphic design and multimedia have suggested two directions for dimensional typography: • ‘normal’letterformsused in increasingly complex informationlayering(for example, the work of Muriel Cooper, MIT Media Labs), and • exploring the ‘sculptural’and ‘three dimensionalforms’ ofindividual letters (for example the work of J. Abbott Miller)

11. the work of the Visible Language Workshop at MIT recreates an information landscape where clusters of text-based information objects are scattered throughout a three dimensional space on the computer display the user flies through a textual three space which provides a kind of visual context so it is difficult to get lost Overview at the Massachusetts Institute of Technology Letters and Information LayeringVLW at MIT (Muriel Cooper & David Small)

12. this kind of work has inspired further developments for example the kind of information presented in the table of contents for the MIT Bulletin by David Small and Suguru Ishizaki (left) Note that different levels of abstraction (general versus specific) are coded according to size, colour and orientation- this kind of coding redundancy is extremely useful for human readers Letters and Information LayeringCoding Redundancy

13. a direct application of this kind of idea to organisational multimedia is the experimental, interactive information space called financial viewpoints- these kinds of displays will likely make their way very quickly into high-end executive information systems (EISs) developed by Lisa Strausfeld to provide a volumetric representation of a sample portfolio of seven separate mutual funds Letters and Information Layering‘Financial Viewpoints’ Lisa Strausfeld

14. Letterforms and Computation • obviously the kind of information layering occurring in these ‘information landscapes’ are the result of computationon a vast scale… • … but individual letterforms are also subject to computation- this simply the latest technological intervention in the design of letters and fonts and we now turn our attention to consider this less obvious aspect of the ‘sculptural forms of letters’ • as we shall see, the design of letters and fonts has always been in response to technological opportunities and economic requirements…

15. Sculptural Forms of LettersBased on J. Abbott Miller (1996) • there has been continuous experimentation with letterforms- leading to the development of new type faces • readers and viewers are increasingly able and willing to navigate texts and negotiate challenging textual and visual environments either in physical exhibitions or the virtual environments • designers accustomed to dealing with the flat, pictorial paradigms of print are now dealing with architectural, ergonomic, and cinematic paradigms of environmental, immersive media • two of the most common ways that are used to create new type faces and fonts are extrusion and rotation- we then describe several less frequently used approaches

16. Most programs- including this one- automatically dimensionalise fonts by extruding them and rendering them in simulated stone, glass and other textures Extrusion is the most common technique for converting a flat letter form into an object Sculptural Forms of LettersExtrusion Extrusion Extrusion

17. Sculptural Forms of LettersRotation… • like extrusion, rotation of letterforms produces classical forms spheres, columns and cones • by transforming the signature silhouette of a letter into a solid- and often closed form- it generates less recognisable letterforms and is therefore rarely seen • if it is attempted it is usually rotation along the horizontal rather than the vertical axis

18. Sculptural Forms of LettersRotation… • shown below is the Univers font designed by Adrian Frutiger (1957)- a highly structured san serif font • in a homage to this most widely used fonts, Ji Byol Lee (1996) designed the vertical rotation of Univers called Univers Revolvedsee next slide…

19. Sculptural Forms of LettersUnivers Revolved font (Ji Byol Lee, 1996)

20. Sculptural Forms of LettersTubing, Shadowing… • there are many other types of transforms that are familiar and so generally go unrecognised, including: • tubing which is related to extrusion and rotation but the operation is limited to the stroke of the letter rather than the shape of the overall character • shadowing evokes three dimensionality through the use of implied light sources an example is the typeface called ‘Umbra’ (below)

21. Sculptural Forms of Letters … Sewing, Bloating • Other types of deformation on letterforms that are used for creating new type faces include: • sewingrefers to two dimensional letters which resemble ribbons- also those type faces that are created with reference to stitching, threading, and lacing • bloating refers to operations produce letters which Miller (1996, p. 7) describes as “… bulbous, organic, corpulent, inflated, and biomorphic in nature… reminiscent of both the way skin envelopes a skeleton understructure, and the shapes produced by the expansion and contraction of membranous materials”

22. …Sculptural Forms of Letters Molecular & Modular Construction • also there are ways in which letters can be formed which rely directly on fabrication or construction metaphors: • Molecular Construction involves building individual letters from similar, smaller units (like using brick-like blocks to build a letter) • Modular Construction involves producing letters from a small set of interchangeable parts (eg. if we were to use special construction symbols to form to shapes of letters)

23. Sculptural Forms of LettersRelationship to Economics & Technology… • the history of font and letterform design has been in response to: • economic needs- the design of typefaces that are best suited for the display of notices for use in advertising (eg. Ironwood, Juniper, Mesquite fonts are designed for exactly these purposes), and/or • modes of production- eg. the actual type of the above fonts was made of wood rather than iron- a cheaper material, and/or • inspired by and in some cases mimicking technological developments (eg. Rubberstamp, Bouchon, Magatama, LCD, and Synchro)

24. Ironwood(1990) Juniper(1990) Mesquite(1990) Letraset Rubber Stamp (1983) by British designer Alan Birch to simulate a stencil, branding or rubber stamp effect. Sculptural Forms of LettersRelationship to Economics & Technology… All of three of these fonts have been designed by Kim Buker, Barbara Lind, and Joy Redick (1990) as a revival of a wood decorative fonts popularly used during the industrial revolution

25. Bouchon typeface designed by French husband and wife design team, Roselyne & Michel Besnard is influenced by bitmapped letter images Designed by eminent Japanese designer, Hajime Kawakami, Letraset Magatama has a design influenced by computer 'LCD' typeface named after 'Liquid Crystal Display' technology inspired British designer Alan Birch (1991) to create this typeface ‘Synchro’ was designed by British Designer Alan Birch (1984) it uses a look that emulates dot matrix printers Sculptural Forms of Letters… Relationship to Economics & Technology

26. Sculptural Forms of LettersText under the influence of Computation … • therefore, it should not be surprising that the computer simply becomes yet another tool which: • can be usedto create new fonts and font families and/or • can be used in a self-reflexive manner- to make text that obviously looks like computer graphics • for multimedia developers, these kinds of explorations in the relationship betweenmedia and computation are very important- they provide us with insights into the nature and the function of media and also act as new exemplars for use in our own development work

27. the alphabet for a modern fancy font called Dr. No-b, developed by Ian Andersen (1992) is shown above while these letters are reasonably legible when small they become a little confusing when enlarged the lower case ‘a’ is shown on the left hand side Sculptural Forms of Letters… Text under the influence of Computation …

28. Sculptural Forms of Letters… Text under the influence of Computation … Dr. No-b (above left) can be transformed into a connected dot structure (above right) and then displayed as a matrix of spheres in which individual letters are formed by viscous, lava-lamp-like linkages. The resulting effect on the right hand side is called Mercury

29. based on a 15th Century round gothic capital ‘S’ formed from a series of circles, the Tapeworm font, from which this example is taken, interprets the circle as a sphere on which the letter is wrapped the computer graphics technique of wrapping textures or flat images onto solids is commonly used in advertising- here we see it in the service of typography Sculptural Forms of Letters… Text under the influence of Computation …

30. Sculptural Forms of Letters… Text under the influence of Computation … • in the next two slides we see fonts- possessing features that could be the result of computation- themselves being transformed from two dimensional shapes to three dimensional objects • the first example uses a font called Modula Ribbed designed by Zuzana Licko (1995). The ‘ribbing’ looks like an iteration over the outline of the two dimensional letterform. Converted to three dimensions the font is referred to as Polymorphous • the second example uses a font called Jesus Loves You designed by Lucas de Groot (1995). The iteration over the outline of the two dimensional letterform is suggestive of the crown- of-thorns like spines from which the font derives its name. Converted to three dimensions the font is referred to as Rhizome after the writings of the French philosophers Deleuze and Guattari.

31. Sculptural Forms of Letters… Text under the influence of Computation … Modula Ribbed Alphabet Modula Ribbed 2D detail lowercase letter ‘f’ Polymorphous 3D detail lowercase letter ‘f’

32. Sculptural Forms of Letters… Text under the influence of Computation … Jesus Loves You Alphabet Jesus Loves You 2D detail lowercase letter ‘f’ Rhizome 3D detail lowercase letter ‘f’

33. Sculptural Forms of Letters… Text under the influence of Computation … • the next slide shows what are called ligatures- forms which are the result of welding two letters together • ligature is defined as: • the act of binding or tying up- the ‘welding’ described above, or alternatively a link, bond or tie- in these examples a possible tie is that they are consecutive letters in the alphabet • this term also has a meaning in traditional printing- it is a character of two or more joined letters such as fi, fl, ffi, ffl • designed by Bart Overly (1995) these forms were devised so that they could be ‘read’ from different orientations in three space- of course not all orientations provide an intelligible reading!

34. Sculptural Forms of Letters… Text under the influence of Computation …

35. Sculptural Forms of Letters… Text under the influence of Computation … • the next and final example of the text under the influence of computation is the positional transformation and interpretation of a font which has thick and thin elements • the letter constructions (left of the next slide ) are based on the classic font Didot designed by François Ambroise Didot (1784) • the thick and thin variations of Didot are interpreted as functions of perspective and the position of planar elements (right of the next slide)

36. Sculptural Forms of Letters… Text under the influence of Computation

37. Dimensional TypographyMerging the ‘Spatial’ and ‘Sculptural’ • these avenues of research within dimensional typography can and will merge into one another • spatial aspects of navigation (based on work like that of Muriel Cooper) merging together with the sculptural aspects of individual forms (based on work like that of J. Abbott Miller) • the goal is to create interfaces which blend, merge and evolve based in user interaction- a new design aesthetic with new functional possibilities!

38. Time Slice Imaging

39. Time Slice Imaging (1) • to capture a time slice image: • use a special camera consisting of a large number of still cameras (~120) and arranged them in a large arc • the optical configuration of each still camera is such that each image overlaps its predecessor and successor • connect the cameras so that they all take an image of the same subject simultaneously

40. Time Slice Imaging (2) • to create the time slice image: • edit together each of the still images either using linear film editing or by using non-linear digital editing, and • assemble the images onto a video tape ordering the images according to camera position- that is in a sequence from left-most camera to the right-most camera

41. Time Slice Imaging (3) • then play back the movie! • the result is a captivating experience- a frozen moment scene from a huge number of angles • it is so startling because we do not see time this way and we are never able to get a view from multiple positions • truly a new media- technically identical to digital video but very different for users!

42. Time Slice Imaging (4)Timeslice Camera “This camera gives a five-metre long 90 degree circular tracking shot in time-slice, live-action, long exposure, high-speed shutter or any combination. The camera performs like a compact motion-control rig. The optics are multi-coated, allowing the camera to perform to wide screen feature film standards. Again the construction is robust, enabling the camera to travel to far flung locations and work under the harshest conditions (as has proven the case with natural history work).” http://www.timeslicefilms.com/cameras_pc.html

43. Time Slice Imaging (5)Examples • first mentioned in a small news item- Scientific American • Other Examples: • Various Advertisements • Lost in Space (1998) • The Matrix (1999) • Lecture Video Example: • BBC (1998) The Human Body- Part 2

44. Time Slice ImagingCreating a New Media: STI • if you understand media you are in a better position to create entirely new kinds of passive and interactive experience for users, • a hypothetical image capture system is proposed here- a variation on Time Slice Imagery which we will call Space Time Imaging (STI) • for example: • replacing the arc of still cameras with a sphere of digital video cameras would produce on playback the kind of imagery that could freeze both space and time independent of each other • the observer of an STI sequence of a football player kicking a ball would be able to see the action from unusual points of view, eg. from the point of view of the ball, or footballers toe!

45. Virtual RealitiesPartially Immersive

46. Virtual RealitiesAn example of the relevance of new media • some new digital media are being used to solve organisational problems • for example Vitual Reality is being used by: • Real Estate Agents who use it to describe the interiors of expensive properties which are to be auctioned, • Queensland Police are using it to create a realistic reconstruction of the crime scene

47. Virtual RealitiesTwo Major Classes • Cylindrical Virtual Reality- in which a set of images is stitched together to give on playback the user appears to sit in the centre of a cylindrical projection. Permits only a limited vertical field-of-view- in practice this is not a disadvantage • Cubic or Spherical Virtual Reality- in which either: • (i) a fisheye lens is used to create a highly distorted view of an image or • (ii) a number of strips of images are stitched together to form a sphere in which on playback the user appears to sit in the middle of a sphere. They totally enclose space allowing the user to look straight up or down as well as around

48. Virtual RealitiesImportance of QuickTime VR • we will concentrate on QuickTime VR which was the first partially immersive VR system • QTVR is proprietary in that it must be developed on a high-end Macintosh, • but can be played on multiple platforms just like QuickTime • not a problem for multimedia developers who often prefer this platform because of its continuous support of graphic arts and design markets

49. Virtual RealitiesSpatial Classification of VRs • there are several types of VR that can be built using QuickTime VR: • Object Movies • Single Node Panoramic Movies • Multi-Node Movies • Sparse Multi-Node Scene • Continuous Multi-Node Scene

50. Virtual RealitiesSources of Object Movies • photography of real object/s from all views using film or video • model and render virtual object/s from all views digitally • each view becomes a distinct frame in a frame space formed by a QuickTime movie