1 / 61

Generating Virtual Webpages

Explore the concept of virtual webpages, which are generated on the fly and tailored to individual user profiles and interaction contexts. Learn about information integration, media objects, and the role of ontological annotations. Discover how AI technology can enhance user interfaces through personalization.

jkell
Download Presentation

Generating Virtual Webpages

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Generating Virtual Webpages EASSS 2000 2nd European Agent Systems Summer School Monday, August 14 2000 Wolfgang Wahlster German Research Center for Artificial Intelligence, DFKI GmbH Stuhlsatzenhausweg 3 66123 Saarbruecken, Germany phone: (+49 681) 302-5252/4162 fax: (+49 681) 302-5341 e-mail: wahlster@dfki.de WWW:http://www.dfki.de/~wahlster

  2. Three Generations of Web Sites First Generation Second Generation Third Generation Virtual Webpages Interactive Web Sites Static Web Sites Netbots, Information Extraction, Presentation Planners JavaScripts and Applets User Modeling, Machine Learning, Online Layout Database Access and Template-based Generation Fossils cast in HTML Dynamic Web Sites Adaptive Web Sites

  3. What is a Virtual Web Page? Virtual Memory, Virtual Relation, Virtual Reality... A Virtual Web Page (Wahlster 1998) lis generated on the fly as a combination of various media objects from multiple web sites or as a transformation of a real web page. l looks like a real web page, but is not persistently stored. l integrates generated and retrieved material in a coordinated way. l can be tailored to a particular user profile and adapted to a particular interaction context. l has an underlying representation of the presentation context so that an Interface Agent can comment, point to and explain its components.

  4. AiA: Information Integration for Virtual Webpages PAN Travel Agent Andi Car Route Planner Yahoo News Server Yahoo Weather Server Gault Millau Restaurant Guide Hotel Guide

  5. Virtual Webpage Retrieved from 5 Different Servers

  6. The Combination of Retrieved and Generated Media Objects for Virtual Webpages Multi-Domain Problem Specs NETBOT Multiple Data Sources Distributed Information Information Structures l Relations, Lists l KR Terms Media Objects l Texts, Sounds, Videos l Pictures, Maps, Animations Retrieved Results

  7. The Combination of Retrieved and Generated Media Objects for Virtual Webpages Information Structures l Relations, Lists l KR Terms Media Objects l Texts, Sounds, Videos l Pictures, Maps, Animations Retrieved Results Select Canned Media Objects Design New Media Objects Coordinate Media Objects Transform Media Objects l Icons, Clip Art l Frames, Sounds l Graphics, Animation l Text, Speech, Mimic l Temporal Synchroni- zation l Spatial Layout l Clip, Convert, Abstract l Zoom, Pan, Transition Effects Select & Design Reuse & Transform

  8. Operational Models of Referential Semantics for Robots and Netbots (Wahlster 1999) Robot Netbot Set of Subsumption Relations in an Ontology Set of Subsumption Relations in an Ontology “Screw” “Departure Time” Set of Recognizers Set of Wrappers Physical Objects WWW Objects ... ... Screw 1 Screw N DT 1 DT N

  9. Enhancement of User Interfaces through Personalization System is able to flexibly tailor presentations to the individual user and the current situation. An animated character serves as “Alter Ego” of the presentation system. Personalized Presenters at DFKI

  10. The Role of Ontological Annotations for the Generation and Analysis of Virtual Webpages (Wahlster 1999) Webpages with Ontological Annotations Webpages without Ontological Annotations Information Extraction Agent Presentation Planner With Ontological Annotations in: SHOE, OML, XOL,OIL, DAML and Persona Annotation in PML Virtual Webpage Information Extraction Agents TriAS Presentation Agent Persona

  11. Action Planning in Robotics Speech Act Theory Philosophy of Language/Linguistcs Artificial Intelligence Plan-based NL Dialog Systems Gestural Acts and Mimics Graphical Acts Computational Linguistics Semiotics Semiotics Plan-based Multimodal Interface Agents Artificial Intelligence Towards Plan-based Multimodal Interface Agents

  12. The Generation of Virtual Webpages with PAN and AiA Address Hotel Agent Map Agent AiA Presentation Planner Pictures and Graphics Netbot PAN Pieces of Text Components of virtual Webpages Virtual Web Presentation Coordinates for Pointing Gestures Trip Data Input for Speech Synthesis Persona Server Icons for Hyperlinks Constraint- based Online Layout Weather Agent Train & Flight Scheduling Agent Major Event Agent

  13. Persona as a Personal Travel Consultant

  14. AIA’s presentation planner (Andre) has been extended to accommodate for various target platforms through the introduction of a mark-up language layer Presentation Planner PET- PML Agent Script WML SMIL PET Persona Player WML-Browser MS-Agent Controller SMIL Player

  15. PPP’s Persona Server implements a generic Presentation Agent that can be easily adapted to various applications Visual Appearances Behaviors Hand-drawn Cartoon l Presentation Gestures Bitmaps l Reactive Behaviors l Idle-time actions l Navigation actions Persona Server Auditory Characteristics Video Bitmaps l Sound effects, auditory icons l Voice: male, female Generated Bitmaps from 3D-Models

  16. Talking Posture 1 cautious, hesitant appeal for compliance avoids body-gestures Talking Posture 2 active, attentive self-confident uses body-gestures Classification of Persona Gestures(Andre, Baldes, Rist) Gesture Catalogue

  17. Context-Sensitive Decomposition of Persona Actions High-Level take-position (t t ) point-to (t t ) 1 2 3 4 Persona Actions Context-Sensitive Expansion move-to (t t ) r-stick-pointing (t t ) 1 2 3 4 (including Navigation Actions) Decomposition r-turn (t t ) 1 21 r-hand-lift (t t ) into 3 31 r-step (t t ) Uninterruptable 21 22 Basic Postures f-turn (t t ) r-stick-expose (t t ) 22 2 31 4 Bitmaps ... ... ... ...

  18. Extensions of the Representation Formalism Distinction between production and presentation acts (i.e. Persona- or display acts) Explicit representation of qualitative and quantitative constraints Production Act Presentation Act Introduce S-Show S-Position Elaborate-Parts Create- Graphics S-Wait Label Label S-Create- S-Depict Window S-Speak S-Point S-Speak S-Point Qualitative constraints: Create-Graphics meets S-Show, ... Metric constraints: 1 <= Duration S-Wait <= 1, ...

  19. Persona Presents an Automatically DesignedBusiness Chart

  20. Virtual Webpage with Animation Effects Based on a Single GIF Image

  21. Transition Effects in a Series of Retrieved Pictures

  22. PET: Persona-Enabling Toolkit (Müller, Neurohr) Objective: l Enable non-professional computer users to populate their web pages with lifelike characters PET comes with: l a set of characters and basic gestures l an easy-to-learn Persona markup language Developer’s PET will include: la character design tool which enables users to build their own characters Technical Realization: l Based on XML and Java

  23. The Persona Markup Language Specification of the character to be used Specification of Persona actions <html> <head> <title> Persona Test </title> </head> <body> <persona bitmap=“cartoon” ...> <uselib url= .../> <do name=“greet”/> <do name =“speak” args=“hello”/> </persona> </body> </html> Features: • XML-based • easy to learn

  24. Tasks of PET PET-Parser l Preprocessing (required for non-standard HTML pages) l Identification of Persona Tags • PET-Generator • l Replacement of Persona Tag with Java Applet • l Determination of parameters for Java Applet • l Realization of Persona active elements as JavaScript • l Generation of DHTML layers (transparency of Applet box) • l Generation of data, e.g. audio files for speech output • l Generation of scripts

  25. Functional View of PET PET Parser PET Generator Behavior Monitor Event Handler Character Composer Bitmaps Webpage with Reference to Java Applet URL of Webpage with Persona Tag <html> ... <APPLET archive=“personaplayer.jar”...</APPLET> ...</html> PET Application Server <html> <head> <title> Persona Test </title> </head> <body> <persona bitmap=“cartoon” ...> <uselib url= .../> <do name=“greet”/> <do name=“standard”/> <do name =“speak” args=“hello”/> </persona> </body> </html> Persona Scripts waitscreen 4 gesture greet 0 0 null gesture laugh 0 0 null ... Persona Engine Audio Data

  26. The Bidirectional Control Flow onPersona-Enabled Webpages l Mouse Clicks l Mouse Movements Triggers actions of the Persona l Text Input l Speech Input l Menu Input l Direct Manipulation Input Triggers operations on elements of the webpage Web Persona

  27. Persona Active Elements (PAE) <ACTIVEIMAGE SRC=“image” HREF=“url” NAME=“image name” STATUS=“status message” ALT=“tooltip” CACTION=“persona action onClick” OACTION=“persona action on MouseOver” ...> <PDIV DIVNAME=“name of the element” DVFRAME=“frame name” TOP=“anchor-y” LEFT=“anchor-x”>some HTML elements</PDIV> • l Active Images • An active image starts a persona action when clicked. • l Addressable Objects • An addressable object is an object which can be addressed and • manipulated by Persona via its name and its position.

  28. Multimedia coordination in previous systems: no declarative representation of spatial and temporal layout no synchronization of Persona actions with other dynamic multimedia objects Solution: Use PrePlan for automated generation of SMIL expressions Synchronized Multimedia Integration Language officially recommended by W3 consortium Spatial and Temporal Coordination of Multimedia Presentations

  29. High-level specification of temporal and spatial constraints within the operators of the Presentation Planner PrePlan Use of the incremental constraint solving toolkit Cassowary Uniform treatment of spatial and temporal constraints Transformation of qualitative constraints into metric constraints Integration of a backtracking mechanism to handle disjunctions Approach

  30. Processing Steps Decomposition of presentation goals into elementary acts and collection of spatial and temporal constraints Constraint propagation using extended Cassowary constraint solver Representation of spatial and temporal layout as SMIL constructs

  31. (define-plan-operator :header (A0 (ShowPresentation ?topic)) :constraints (*and* (BELP (Illustrates ?video ?topic)) (BELP (Video ?video)) (BELP (Sets-to-Music ?audio ?topic)) (BELP (Audio ?audio)) (BELP (Summarizes ?title ?topic))) :inferiors (A1 (SAddSmilCode (?video))) (A2 (SAddSmilCode (?audio))) (A3 (SAddSmilCode (?title))) :temporal ((A1 (d) A3) (2 <= begin A1 - begin A2)) :spatial ( (aligntop A1) (centerh A1) (centerh A3) (20 <= top A1 - bottom A3 <= 20)) Example of a Plan Operator

  32. Spatial constraints Qualitative constraints (Image1 LeftOf Image2), (Image1 TopAlign Image2) Quantitative constraints (4 < Top Image1 - Top Image2) Temporal constraints Qualitative constraints Allen constraints: (meets, before, overlaps, ...) Quantitative constraints Linear inequalities: (Begin Audio1 - End Audio2 < 3), (2 <= Duration Audio1 <= 6) Representation of Spatial and Temporal Constraints

  33. <smil> <head> <meta content="Smiling Saarland" name="title"/> <layout> <root-layout height="50" width="300"/> <region height="30" id="reg43463" left="0" top="0" width="300"/> </layout> </head> <body> <par> <audio begin="1.0s" end="21.807s" src="snowman.wav"/> <textstream begin="0.0s" end="22.807s" region="reg43463" src="snowman.rt"/> <video begin="3.0s" end="21.807s" src="snowman.avi"/> </par> </body> </smil> Resulting SMIL-Document

  34. Some Examples: point to an object when it appears in a video comment on a video 3 seconds after it has started Smile when video appears on the screen repeat lip movements until audio stops Synchronization of Persona Actions with other Media Objects (Andre, Kleinbauer)

  35. (define-plan-operator :header (A0 (ShowPresentation ?topic)) :constraints (*and* (BELP (Describes ?audio ?topic)) (BELP (Audio ?audio)) (BELP (TalkingGesture ?video)) (BELP (RepeatGesture ?video)) :inferiors (A1 (PresentPictureSequence (?topic))) (A2 (SAddSmileCode (?audio))) (A3 (SAddSmilCode (?video))) :temporal ((A2 (d) A1) (2 <= begin A2 - begin A1) (A2 (e) A3)) :spatial ( (aligntop A1) (alignleft A1) (1 <= bottom A0 - bottom A3 <= 1) (1 <= right A0 - right A3 <= 1)) Synchronization of Persona Actions

  36. <smil> <head> <meta name="title" content="Cyberella in RealPix Demo" /> <layout> <root-layout width="650" height="385" background-color="#9898F8"/> <region id=“reg471101" top="0" left="12" width="385" height="284" background-color="#9898F8"/> <region id=“reg081591" top="95" left=“354" width="295" height="385" background-color="#9898F8"/> </layout> </head> <body> <par> <img region= " reg471101" begin= " 0.0" src= "schwarzenacker.rp" fill="freeze"/> <animation begin= "2.0s" end="15.1" region="reg471101" src= "talking-gst.rp"/> <audio begin= "2.0s" end="15.1" src="schwarzenacker.rm" /> </par> </body> </smil> SMIL Specification for Persona Presentation

  37. Resulting Timeline Diagram ... ... ... At archeological finds in Schwarzenacker, ... ... ...

  38. Using SMIL to Synchronize Persona Actions

  39. Information Filtering Information Retrieval Information Integration identify relevant documents wrappers Information Extraction Agents • ... • identify and extract relevant • pieces of information • transform them into canonical form • wrappers • operational descriptions of a target concept • abstract from concrete occurrence within document • robust against modifications

  40. The Trainable Information Agents Framework (Bauer, Dengler) combination of "classical" problem-solving methods and information agents specifications preferences/heuristics Application planning knowledge user preferences results info requests info domain ontology requests User Browser InfoBroker improved dialog guidance training site info/update query planning, optimization, and execution info requests info or script Web site annotations site information Info Extraction Trainer PBD dialog

  41. High Degree of Parallelism of Queries

  42. common part (usable for communication) Naive User Learning Annotation Agent Knowledge about a Webpage Shared by User and Agent visual/semantic structural procedural

  43. Train_Connection [ from =>> Location; to =>> Location; travel_date =>> Date; time =>> Time; depart_time =>> Time; arrive_time =>> Time; cost =>> Price; travel_duration =>> Duration; info_url =>> URL; ... ] Example - Ontology

  44. states: information states concepts / attributes and instantiations operators: querying schemes preconditions (´+´) and effects (´-´) to time arrive_time travel_duration < +, +, +, +, -, -, -, u, - > from travel_date depart_time cost info_url Query Planning - I

  45. Ù = S Int ( op ) 0 State Ontology Top City.value = München City.language = German ... String ... Operators CityName1 Language – – < String, Language, String, Language > + + City babelfish . . . value language ¢ c c ( ) " Î ¢ $ = Ù c pre op i : S i :: c 0 Query Planning - IV

  46. Query Plan Visualization • Features • alternative queries • past states • future states • state descriptions • PBD requests • accept / reject PBD request • assessment of plans • expected completion time

More Related