Multimedia Pioneering Creating Mobile And Location-Based Applications

1. Multimedia PioneeringCreating Mobile AndLocation-Based Applications IMM INFO59724 Richard Ballard www.InSeed.com 416.414.8100 Quick Plug... www.GODSPAWN.com

2. Motorola Compaq Siemens Panasonic Ericsson Nokia Why Is Mobile Special? It’s not just phones. It’s PDAs, videophones, Internet browsing, location-based services, interactive games and a whole lot more….

3. Content Evolution • WAP (“DOS like”) • Text only • Very Slow • Poor layout • xHTML (“DOT COM”) • Hypertext linked • Limited Formatting • Restricted UI • J2ME/BREW/FL (“mTV+iTV”) • Rich • Dynamic • Streaming

4. Mobile is Rich • Images • Still camera, email images • Live images • Audio • Phone/PDA and MP3 player in one • Video • Display moving images This is where it’s at folks!

5. Corporate access Accelerated connections, voice and data multitasking Always-on, performance and security enhancements Dial-up connections Browsing Downloading Video content (MMS, streaming) xHTML for WAP and web integration, animated content Instant connectivity, colour screen, WAP Push Downloadable applications, MIDI sounds Black and white screens, simple graphics Ringing tones, icons, screensavers, business cards Apps Evolve Rapidly Messaging Hi, I’ll see you at 6 PM Regards, Bob Multimedia messaging with video Multimedia messaging with pictures and audio Basic text messaging

6. Many Apps Source ADLittle02

7. Many Apps Source ADLittle02

8. Apps are Multi-Modal • Travel information • Make request via voice • Receive response in text • Directions • Make request via voice • Receive initial response in text • Get updates while traveling via voice or SMS or rich graphics • One-to-many messaging • Record message via voice or text • Deliver message via voice, SMS, WAP, or email • Purchase famous voice for a personal answering message • Text or voice menus • Voice or text to hear message or select and authorize payment • Unified communications • While listening to a voice message from a customer, obtain a text display of recent customer activity • Emergency response team • SMS and voice alert • Conference and text while traveling to emergency • Cricket matches (Hutchinson India) • SMS alert at start of coverage • Live voice coverage or text updates • Information delivery (SFR France) • SMS broadcast with phone # & URL • Choice of text display or voice (text-to-speech) • Yellow pages (Platinet Israel) • Adding voice menus to existing text-based service • Voice flattens menus, eases access

9. App Market Segments • Fun: • WWW, video, post card, snapshots, text, picture and multimedia messaging, datacast, personalisation applications (ring tone, screen saver, desk top), jukebox, virtual companion / pet ... • Work: • Rich call with image and data stream, IP telephony, B2B ordering and logistics, information exchange, personal information manager, dairy, scheduler, note pad, 2-way video conferencing, directory services, travel assistance, work group, telepresence, FTP, instant voicemail, colour fax ... • Media: • Push newspaper and magazines, advertising, classified ... • Shopping: • E-commerce, e-cash, e-wallet, credit card, telebanking, automatic transaction, auction, micro-billing shopping ... • Entertainment: • News, stock market, sports, games, lottery, gambling, music, video, concerts, adult content ... • Education: • Online libraries, search engines, remote attendance, field research ... • Peace of Mind: • Remote surveillance, location tracking, emergency use ... • Health: • Telemedicine, remote diagnose and heath monitoring ... • Automation: • Home automation, traffic telematics, machine-machine communication (telemetry) ... • Travel: • Location sensitive information and guidance, e-tour, location awareness, time tables, e-ticketing ... • Add-ons: • TV, radio, PC, access to remote computer, MP3 player, camera, video camera, watch, pager, GPS, remote control unit ... Any Of These Can Be Location Based Source UMTS Forum describes 11 Service Categories

10. Apps Convergence Source: Nokia 2004

11. Many Devices Latest Nexio PPC: http://www.dynamism.com/nexio/gallery.shtml

12. Developer Challenges • Yesterday • Apps: Voice mail & messaging, access to web via PC • Platform: Closed arch. & proprietary • Today • Apps: Voice, SMS, MMS, wireless access to web via WAP • Market: Macro-segments • Platform: Flexible, closed • Tomorrow • Apps: Voice and data/web convergence • Market: Micro-segments • Platform: Open, scalable, versatile • Market Trends • Profound changes taking place in the industry • TDM -> IP; Fixed -> Mobile; Closed -> Open • Enterprises moving to IP infrastructure & IP-based managed services • Disruptive changes create opportunities • Moore’s Law • Mobility, web paradigm, content access • Multimodality • Business Models Shifting • Traditional solutions companies shift focus to customers, applications and systems integration • New go-to-market challenges • Additional partnerships, and Increased complexity in supply chain • Fewer internal resources, Cost savings, Reduced time-to-market

13. Developer Challenges • Divergent Mobile Operating Systems • J2ME / BREW / SYMBIAN/ WinCE / PalmOS • So develop in Java / C / C++ / Flash ? • Divergent Wireless Networks • GSM, GPRS, CDMAOne, 1xRTT, CDMA2000 • 2G @ 9.6kps, 2.5G @ 25kps, 3G @ 50kps • Divergent Handsets • Capabilities, form-factors, hard/soft interfaces, API’s • 1MIP to 400MIP CPU’s • Redevelop app for each handset? • Divergent Provisioning Systems • What apps are downloadable, online, streamed? • Billing support? (PPV, PPD, Subscribe) • Fragmented wireless market • Limited App portability • Confused App distribution • New Development Models • Open, modular, IP-connected subsystems interfacing via standard protocols…yeah, right!

14. Carrier’s Value Chain Before After During Revenues in Value Chain 2G v 3G CMS Lifecycle Sources: Alcatel and 3GPP 2004

15. Content Value Chain Content rights Content creation Application/Content aggregation Portal Application/Service Hosting Transaction Management/Billing Network Terminal Service delivery • Nokia • Sony/Ericsson • Trium • Siemens • NEC • Panasonic • Samsung • Microsoft • Palm • Compaq • High street retailers • Online retailers Application creation • BBC • Universal • Sony • BMG • Eidos • Time Warner • Bloomsbury • FremantleMedia Platform / Technology provider • Genie • Vizzavi • Zed • T-Motion • Boltblue • Djuice, iobox • EDS • Exodus • Logica • CGE&Y • Digital Bridges • MNOs • Egg • Visa • Mastercard • Vodafone • Orange • mmO2 • T-Mobile • TIM • Telefonica • Sonera • Codeonline • iFone • Ngame • Games Kitchen • Digital Bridges • In-fusio • Terraplay • Picofun • Springtoys Content User Source: Telecom1 UK

16. Distribution Channels • Direct Marketing by Developer, e.g. • Trip Planner: • http://www.teleatlas.com/homepage.swf?hp_counter=02&countryCMS=uk&language=en • Mobile CRM: • http://www.dotcomments.com/ • Mobile Sales Accounting: • http://www.bmobiletech.com/flash/ipaq.swf • Carriers’ via Developer Community, e.g. • Verizon • http://www.vzwdevelopers.com/aims/ • Cingular • https://alliance.cingularinteractive.com/dev/cda/home • Note: carrier wireless technologies: • Analog • TDMA/GSM/iDEN (modem) • CDMA/1x/3xRTT/CDMA2000 (digital) • WCDMA (EDGE, UMTS) (from GSM) • ODFM (Nextel) • Content Aggregators, e.g. • ClearSky http://www.clearskymobilemedia.com/ • Handset Vendors, e.g. • Club Nokia www.nokia.com/clubnokia • Nokia Content Aggregator http://www.preminetsolution.com/

17. Discovery Portal Screen real-estate = discovery bottleneck 1. Operator Portals • Handset-friendly web-sites that can be customized by individual users to reduce click-time to favorite content/services 2. TV Broadcasters • TV is becoming a required competence for 3G Operators, the hottest applications include: event promo, real-time voting shows, subscriptions to advertisers, pre-familiarized audio/video skin downloads, etc • Evening shows impact traffic management models for mobile operators by pushing networks to capacity • Other opportunities for broadcasters envision PNIP (picture-not-in-picture) to customize viewer dialogue • In Europe, mobile is the RETURN PATH for prime-time ITV • 3G/4G will enable DIGITAL MOBILE TELEVISION 3. Handset Vendor Portals • Nokia’s #1 button connects the thin operating system to Nokia-Club, customizable handset-specific interface 4. Traditional Media • Tabloid newspapers, magazines, billboards, flyers, packaging

18. One Device For All Apps?

19. The Wireless Network Source: www.HewlettPackard.com

20. What is MMS? • Enhanced SMS…limited size messages are sent over the control (not the signaling) channels of a GSM’s network. GSM is circuit switched, whereas 3G is an all-IP network. • Non-real-time, multi-media message service • Text; Speech (AMR coding) • Audio (MP3, synthetic MIDI) • Image, graphics (JPEG, GIF, PNG) • Video (MPEG4, H.263) • Will evolve with multimedia technologies • Uses IP data path & IP protocols (not SS7) • WAP, HTTP, SMTP, etc. • Adapts to terminal capabilities • media format conversions (JPEG to GIF) • media type conversions (fax to image) • SMS (2G) terminal inter-working • MMs can be forwarded (w/o downloading) and may have validity period • Addressing by phone number (E.164) or email address (RFC 822) • Extended reporting • submission, storage, delivery, reading, deletion • Supports an MMBox, i.e. a mail box • Optional support of media streaming (RTP/RTSP)

21. VM Application ML Application VM (App) Browser (App) Application Device OS Device OS Device OS Processor Processor Processor How Apps Run On Mobile Devices Three Paths: Native OS • Palm • Symbian • Windows CE • Linux Virtual Machine • Flash / J2ME • BREW • AppForge • Syclo Agentry Markup Language

22. Mobile Languages • J2ME (JAVA) • J2ME is a highly optimized Java runtime environment. J2ME is used in many small consumer products, including smart cards, pagers, and wireless devices. • BREW • QUALCOMM’s BREW (Binary Runtime Environment for Wireless) platform is a thin application execution environment that provides an open, standard platform for wireless devices. The complete BREW solution is controlled and managed by carriers, enabling them to easily get applications from developers to market and coordinate the billing and payment process. Carrier’s BREW-based services will enable consumers to customize their handsets by downloading applications over the air from a carrier’s application download server. • XML • XML is markup language for documents containing structured information. Structured information contains both content (words, pictures, etc.) and some indication of what role that content plays (for example, content in a section heading has a different meaning from content in a footnote, which means something different than content in a figure caption or content in a database table). Almost all documents have some structure. A markup language is a mechanism to identify structures in a document. The XML specification defines a standard way to add markup to documents. • CHTML • CHTML, or compact HTML, is a subset of HTML used for small information devices, such as smart phones and PDAs. CHTML is essentially a pared-down version of regular HTML over the Internet. Because small devices such as cellular phones have hardware restrictions such as small memory, low power CPUs, limited or no storage capabilities, small mono-color display screens, single-character fonts, and restricted input methods (the absence of a keyboard or a mouse), there is a need for this simpler form of HTML. • XHTML • XHTML is the reformulation of HTML as an application of XML. • SMS • Worldwide, SMS (Short Messaging Service) is the most popular data service for mobile phones, e.g. a broad range of SMS-based sports, information, and entertainment titles, using both one- and two-way messaging, each of which generates vast numbers of messages. Many SMS applications also send out real-time web-based reports for usage and billing tracking. • MMS • The Multimedia Messaging Service (MMS) is, as its name suggests, the ability to send and receive messages comprising a combination of SMS with rich media including text, sounds, images, and video, but only to MMS capable handsets (e.g. GSM). • WAP • WAP (Wireless Application Protocol) is a widely used set of protocols that standardize the manner in which wireless devices, such as cell phones and some PDAs, are able to access parts of the Internet, such as e-mail and the Web.

23. Wireless OS Contenders • Symbian • PDAs and smart phones • PalmOS & BeOS • Kyocera, Samsung, etc • Microsoft • PocketPC • Stinger • J2ME • RIM, Motorola, etc • BREW • Qualcomm chipset phones

24. BREW vs. J2ME vs. Flash • BREW • Qualcomm’s Binary Runtime Environment for wireless • Only available to CDMA carriers (i.e. America’s and Asia) • C/C++ based • Digitally signed, secure and clear path to revenue • Certified, true BREW testing • End to End solution • BREW Shop on handset • ADS (App download system for provisioning and DRM) • BDS (BREW distribution system for billing) • Java/J2ME • Mainly for GSM/GPRS carriers • 30k or 64k client app limit • Many versions: • Flash and Flash Lite • Standard J2ME and MIDP (JRE) • J-PHONE JSCL • i-mode DoJa • Vodafone VSCL • KDDI ezplus • Sprint PCS extensions • Write once and debug everywhere • No standard provisioning methods or billing infratstructure • Path to revenue patchy • Immature standard, lots of non-standard options • Other mOS: • Symbian • C/C++ based OOS • Niche market handsets • Microsoft • C/C++ based PocketPC Phone Edition • PDA’s or niche market phones “Download Java” Java Games Article on J2ME versus Flash, (excellent summary): http://www.design-nation.net/en/archives/000453.php Article on BREW versus J2ME: http://uk.builder.com/architecture/web/0,39026570,20266771,00.htm

25. Smart Phones vs. PPC’s • Smart Phones: • Kyocera’s QCP 6035/7135 • Handspring Treo 600 • Samsung Palm Phone • HTC/Orange Stinger phone • Windows Pocket PC Devices • New class of handhelds emerging • Palm sized devices that can run Windows XP OS and all XP applications out of the box • OQO • Antelope • Vulcan • PC-PDAs: • Palm’s Tungsten W & i705 • RIM’s Blackberry 7200 series • HP’s iPAQ PocketPC • Dell’s Axium X5

26. Pocket PC sites • Devices & Apps Blogs • www.flashenabled.com • www.howardforums.com • www.engadget.com • www.gizmodo.com • www.msmobiles.com • Pocket PC: • www.Geekzone.co.nz • www.pocketnow.com • www.PDAGold.com • www.PocketPCThoughts.com • http://www.spbsoftwarehouse.com/?en

27. Device Functionality Evolving • Terminals are becoming Multi-Tasking all-purpose Devices • Camera phones as of mid-2003 outsell digital cameras • 75-million camera phones sold during 2003 (source: www.NetworkWorldFusion.com) • Photo messaging growing rapidly in Europe/Asia • In US, Verizon’s first 3-week trial generated 1-million photo-messages • Functional evolution is turning a single hand-held into a combined: • Phone • Personal Digital Assistant (PDA) • Computer • Television • Pager • Video Conferencing Center • Newspaper • Diary • eWallet (universal wireless credit card) • Memory, Battery, Interface and Connectivity Challenges being overcome • PDA memories have expanded by as much as 8-times in the past 12 months • Processing speeds multiplied by 10-times in the past 12 months • HDTV & 3D screens for mobile devices, e.g. Sharp, OED, organic, flexible, disposable • QuadGSM+EDGE+WiFi+EBT integrated antennae, e.g. Motorola, Qualcomm • Single-chip solution workarounds for unsolved battery constraints

28. Developer Resources • Flash Resources: • Macromedia FL community • http://www.flashdevices.net/links/ • Ideas Blog, e.g. Flash-ipod’s and flash-aibo’s • www.flashenabled.com/ • Flash MX templates when publishing to mobiles: • www.macromedia.com/devnet/devices/articles/sap_ppc_02.html • Check which mobile devices are supported: • www.macromedia.com/mobile/supported_devices/ • Download FlashLite and its CDK: • www.macromedia.com/devnet/devices/flashlite.html • XML-HTML Resources • Tutorial on building x-html for mobiles • www.xml.com/pub/a/2004/04/14/mobile.html • Example almost-LBS website for handhelds • www.mcgill.ca/help/mobile/ • Microsoft • Windows Mobile Developer Center • www.microsoft.com/windowsmobile/developers/default.mspx

29. Macromedia Tutorials • Autodialingwww.macromedia.com/devnet/mobile/articles/tel_url.html on (release) {getURL("tel:9055551212");} • FL Gamewww.macromedia.com/devnet/devices/articles/flashlite_games.html • Chat Roomwww.macromedia.com/devnet/mobile/articles/flash_chat.html #include "../as/CHAT.as" _root.CHAT_createSession(); _root.CHAT_setupRoom(); fscommand ("allowscale", "false");

30. Design Tips • Choose content carefully … only that which is essential. Customize the content specifically for the target user, concise and elegantly organized. • Avoid clutter …plan for space economy. Wild colors and patterns splashed across the small screen is frustrating. Avoid graphics that reduce readability, fit them to one screen. Less use of graphics to reduce download time and local memory footprint. • Minimize page length, and maintain a balance between page depth (level of nested pages) and the length of individual pages. • Avoid scrolling …unless its and e-book, consider quick drill down navigation and hypertext link. • Organize information effectively, andtest your applicationusing PDA/phone-vendor supplied emulators (e.g. http://www.palm.com/devzone/pose/seed.html) and then test them with an actual handheld device. • Use supported character sets, most handhelds use the full set of Latin1 characters. International characters are not yet available as internal fonts on some devices. • Size does matter…Low end Palms are 160x160 pixels, the Pocket PC has a screen resolution of 240 x 320 pixels, some CLIÉ models offer 320 x 480, phones are smaller, but all screen resolutions are improving in typically 1.5-year device-ownership cycles • Scaling reduces the quality of many images, so you generally should avoid using images wider than the device's viewing area. Use "large" images sparingly and only when appropriate, as they not only consume screen real estate, they also consume synchronization bandwidth. • Choose contrasting colors…crisp edges. Images that are anti-aliased rarely convert well. • Designing graphics for small, bitmapped screens is something of an art…! Source http://webdesign.about.com/gi/dynamic/offsite.htm?zi=1/XJ&sdn=webdesign&zu=http://www.palmpower.com/issues/issue199902/avantgotips001.html

31. LBS Apps • Flash Lite Instant messenger http://www.icq.com/icq2go/ • Location-aware app displays artwork info while strolling • http://www.macromedia.com/devnet/devices/articles/invisible_ideas.html • Control remote device apps: • http://www.coraccesssystems.com/movies/companionuidemo.htm

32. LBS Concept • Concept: • Find the position of mobile devices while using them for location-based data display and communication • Advantages: • People introduced to local products and services • emergencies, national security, fleet management, dating, logistics… • People can navigate to locations • People can compare nearby things, services, attractions • Suppliers can analyze traffic, qualify prospects and pre-sell • Local objects, services, products and events can be animated to communicate value • They can also co-relate, e.g. 3-fishing buddies’ PDA’s and a boat-rental location are nearby

33. 911 locator Government mandate Find someone within 50m Advertising Impulse buy, walk past Discounts Telematics Mapping and Directions Weather forecasts Truly local Find things: Restaurants Movie theaters ATMs Gas Stations Significant PrivacyIssues Find happiness ASAP Example LBS

34. LBS Requirements • Client • Low client memory, thus J2ME, MobilePC, BREW, WML, XHTML… • Tolerant of unreliable connections • Works fast despite a slow CPU • Server • Fault tolerant: devices and connections are unreliable • Scaleable (add machines to improve throughput) • Roll over sessions to a working machine • Terminate sessions gracefully • Reasonably fast • Basic building blocks: • An accurate and fast positioning technology • An adequate internet connection • Some location data, e.g. maps, geocoding data, points-of-interest • Example GPS data acquisition application at http://www.arecon.de/

35. Positioning Technologies • Cell Origin (“CellID”) • Triangulation: • TDOA (Time Difference On Arrival, measured in phone), • E-OTD (Enhanced Observed Time Difference, measured in network), • AOA (Angle of Arrival, related to CellID) • GPS (geographic position satellite) • Available today • Galileo (EEC 2007) • AGPS (requires surface carrier) • Rosum Digital TV based triangulation • requires surface carrier • Multimode handsets with microcellular underlay • Requires public or private LAN, e.g. BT, WiFi-hotspot, WiMax

36. Typical GPS Data Stream $GPGGA,213104.006,3440.7106,N,08250.6989,W,1,06,01.4,00195.6,M,-32.0,M,,*5B >>> P3 Parking Lot $GPGGA,213105.006,3440.7106,N,08250.6989,W,1,06,01.4,00195.8,M,-32.0,M,,*54 >>> left P3 Parking Lot $GPGGA,213106.006,3440.7062,N,08250.7033,W,1,06,01.4,00202.8,M,-32.0,M,,*50 ... $GPGGA,213333.007,3440.7862,N,08250.5685,W,1,03,02.8,00194.6,M,-32.0,M,,*5C $GPGGA,213334.007,3440.7856,N,08250.5684,W,1,04,03.5,00196.0,M,-32.0,M,,*52 >>> Death Valley Information Corner $GPGGA,213335.007,3440.7851,N,08250.5684,W,1,04,03.5,00196.1,M,-32.0,M,,*55 >>> (cor. Williamson, Avenue of Champions) $GPGGA,213336.007,3440.7852,N,08250.5684,W,1,04,03.5,00196.3,M,-32.0,M,,*57 ... $GPGGA,213614.008,3440.7096,N,08250.3187,W,1,06,01.1,00224.3,M,-32.0,M,,*51 $GPGGA,213615.008,3440.7091,N,08250.3192,W,1,05,01.2,00224.4,M,-32.0,M,,*54 >>> Dan Zen’s Pizza Parlor $GPGGA,213621.008,3440.7070,N,08250.3183,W,1,04,02.1,00227.6,M,-32.0,M,,*5C ... $GPGGA,213849.009,3440.6508,N,08250.2417,W,1,04,02.6,00228.6,M,-32.0,M,,*57 $GPGGA,213850.009,3440.6501,N,08250.2412,W,1,03,03.4,00225.3,M,-32.0,M,,*5F >>> Rhodes Center $GPGGA,213853.009,3440.6501,N,08250.2414,W,1,04,03.7,00228.4,M,-32.0,M,,*54 ... $GPGGA,214049.009,3440.5543,N,08250.3486,W,1,03,02.1,00221.8,M,-32.0,M,,*5A $GPGGA,214050.009,3440.5539,N,08250.3491,W,1,03,02.1,00221.8,M,-32.0,M,,*59 >>> EIB (Fluor Daniel) $GPGGA,214051.009,3440.5541,N,08250.3489,W,1,03,02.1,00221.8,M,-32.0,M,,*5E ... $GPGGA,214218.010,3440.6113,N,08250.4686,W,1,04,01.5,00223.5,M,-32.0,M,,*5C $GPGGA,214219.010,3440.6110,N,08250.4698,W,1,04,01.5,00223.5,M,-32.0,M,,*51 >>> South Palmetto, cor Williamson $GPGGA,214220.010,3440.6110,N,08250.4698,W,1,05,01.3,00223.5,M,-32.0,M,,*5C

37. Decomposing the Data >>> “Dan Zen’s Pizza Parlor” $GPGGA,213614.008,3440.7096,N,08250.3187,W, 1,06,01.1,00224.3,M,-32.0,M,,*51 82˚50.32’ W Longitude 34˚40.71’ N Latitude UTC / GMT GPS fix #satellites altitude (m) So, armed with GPS coords and some location data, all we need is an interface….

38. Dan Zen’s Pizza Parlor

39. The Greasy Weasel Diner

42. Tim Hortons

43. Campus Services Locator http://www.mississaugaicedogs.com/arenamap.html

44. Fun And Games • Games can also be LBS, e.g. • Win points for a real-time Point Of Sale discount • Quiz related to POS-featured product • Skill test to qualify for promotions • Games requiring location visits (e.g. pub crawl) • Inventory info with electronic scratch and win • http://www.dcs.napier.ac.uk/~bill/books.swf • Onsite entertainment for kids at Pizza store • www.ballshooter.com/burgerama/images/b_anim_320x240.gif • Learn the course and safety rules on bike trails • http://www.omnigsoft.com/images/products/MiniSportsbike/tile.jpg • Local showtimes with pre-movie advertainment • http://www.zango.com/downloads/zangoshowtimes.aspx • …And if you can’t get anyone to buy it, everyone loves WAREZ: • http://www.webprostor.cz/pocitace_a_internet/pocketpc/ipaq.swf