1 / 23

Application-specific constraints for multimedia presentation generation

Application-specific constraints for multimedia presentation generation. Joost Geurts, Jacco van Ossenbruggen and Lynda Hardman CWI Amsterdam email: Joost.Geurts @cwi.nl. Talk overview. Generating multimedia automatically Cuypers multimedia generation engine Multimedia and constraints

bobby
Download Presentation

Application-specific constraints for multimedia presentation generation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Application-specific constraints for multimedia presentation generation Joost Geurts, Jacco van Ossenbruggen and Lynda Hardman CWI Amsterdam email: Joost.Geurts@cwi.nl

  2. Talk overview • Generating multimedia automatically • Cuypers multimedia generation engine • Multimedia and constraints • Quantitative constraints • Qualitative constraints • Cuypers demo • Conclusion, future directions

  3. Generating adaptive multimedia • Content • Large multimedia database • System profile • PC, PDA, WAP • Network profile • Modem, Gigabit • User profile • Language, Interests, Abilities, Preferences Too costly to author manually

  4. Cuypers multimedia generation engine

  5. Automatic multimedia generation • Designer does not specify complete presentation……but only specifies requirements • System automatically finds a solution which meets requirements • How should the requirements be specified? • Declarative constraints

  6. Traditional use of constraints • Constraint solving used for problems with: • Many variables • Large domains • Based on domain reduction paradigm • Quantitative constraints • Integer domain • Reduction by arithmetic relations • Greater than (>) • Less than (<) • Equals (=)

  7. Drawbacks of quantitative constraints • Too many (trivial) solutions that differ by: • 1 pixel position, or • 1 milliseconds in timing • Not sufficiently expressive e.g. cannot specify “no overlap” constraint • Too low level e.g. A.X2  B.X1

  8. Solution: qualitative constraints • Example “A not overlap B”, “B after C” • Advantages: • More intuitive • More expressive • Smaller domains • Qualitative solutions translate automatically to lower level quantitative constraints • New problem: What if constraints are insoluble?

  9. Solution: Constraint Logic Programming • Combine Prolog unification and backtracking with constraint solving • Use Prolog rules to generate constraints • Backtrack when constraints are insoluble

  10. Cuypers generation engine • Multiple layers: • Prolog rules to generate constraints • Qualitative constraints translate to quantitative constraints • Solution of both constraints provides sufficient information for final presentation

  11. Cuypers demo: scenario • Client:User is interested in Rembrandt and wants to know about about the “chiaroscuro” technique • Server: Query database • Server: Generate constraints according to: • System profile • User profile • Network profile • Server: Solve constraints / revise constraints • Server: Generate SMIL presentation • Client: Play presentation

  12. Conclusions • Quantitative constraintsare insufficient for automatic multimediapresentation generation. Also need • Qualitative constraintsto allow intuitive and effectivehigh level specification, and • Backtrackingfor revising specific constraintsthat cause the entire set to fail

  13. Future directions • Best-first instead of depth-first • Choose “best” among possible solutions • Needs evaluation criteria • Improve knowledge management • Make design knowledge declarative and explicit • Preserve metadata in final presentation • Use standardized and reusable profiles

  14. Need to make trade-offs • Semantics • Convey message • Aesthetics • Clear / nice layout • Resources • Screen size, bandwidth • Dimension may result in conflicting goals

  15. Quantitative Constraints % csp(+Ids, -Boxes) csp([IdA,IdB],[box(IdA,[x1:AX1, …]), box(IdB,[x1:BX1,…])]) :- % get values maxX(MaxX), maxY(MaxY), height(IdA,HeightA), widtht(IdA,WidthA), … % define domains [AX1,AX2,BX1,BX2]::[0..MaxX], [AY1,AY2,BY1,BY2]::[0..MaxY], % set width & height AX2 – AX1 #= WidthA, AY2 – AY1 #= HeightA, … % constraints AX2 #< BX1, % A left-of B AY1 #= BY1, % A top-align B, … true.

  16. Multimedia and Constraints • Constraint Logic Programming • Domain reduction • Backtracking • Unification (matching rules) • Qualitative Constraints • Non-integer domain • Allen’s 13 temporal interval relations in three dimensions

  17. Qualitative Constraints • Example: • Two images, A,B • A left or right of B • A not above or below B

  18. Qualitative Constraints % csp(+Ids, -Graph) csp([IdA, Idb], [edge(IdA,IdB,x,NoOverlap),…]) :- % define domains NoOverlap :: [b,b-,m,m-], Overlap :: [d,d-,s,s-,f,f-,e], % constraints edge(IdA,IdB,x,NoOverlap), % B not-overlap A edge(IdA,IdB,y,Overlap), % B overlap A true.

  19. Qualitative Constraints • Reasoning • Inverse: edge(A,B,D,Value) <=> inverse(Value,RValue),edge(B,A,D,RValue). • Equality edge(A,B,D,V1), edge(A,B,D,V2) => V1 #= V2 • Transitive edge(A,B,D,VAB), edge(B,C,D,VBC) => tr(VAB,VBC,VAC), % rule generation algorithm edge(A,C,D,VAC). • Translation rules to quantitative domain edge(A,B,D,b) => node(A,D/2,V2), node(B,D/1,V1) V1 #< V2.

  20. Problems in generating multimedia • Text documents are flexible • Add page, scrollbar, • Template models • Wrap text around images • Multimedia documents are less flexible • No pages or scrollbars, no line-breaking or hyphenation • Not based on text-flow • Feedback needed • Linear process model does not work for multimedia

  21. Quantitative Constraints • Example: • Two images, A,B • A left-of B • A top-align B

  22. Cuypers generation engine • Rhetoric/Semantic • Sequence, Example • Communicative devices • Bookshelf, Slideshow • Qualitative Constraints • A before B • Quantitative Constraints • A.X2 < B.X1 • Presentation • SMIL

More Related