230 likes | 355 Views
Application-specific constraints for multimedia presentation generation. Joost Geurts, Jacco van Ossenbruggen and Lynda Hardman CWI Amsterdam email: Joost.Geurts @cwi.nl. Talk overview. Generating multimedia automatically Cuypers multimedia generation engine Multimedia and constraints
E N D
Application-specific constraints for multimedia presentation generation Joost Geurts, Jacco van Ossenbruggen and Lynda Hardman CWI Amsterdam email: Joost.Geurts@cwi.nl
Talk overview • Generating multimedia automatically • Cuypers multimedia generation engine • Multimedia and constraints • Quantitative constraints • Qualitative constraints • Cuypers demo • Conclusion, future directions
Generating adaptive multimedia • Content • Large multimedia database • System profile • PC, PDA, WAP • Network profile • Modem, Gigabit • User profile • Language, Interests, Abilities, Preferences Too costly to author manually
Automatic multimedia generation • Designer does not specify complete presentation……but only specifies requirements • System automatically finds a solution which meets requirements • How should the requirements be specified? • Declarative constraints
Traditional use of constraints • Constraint solving used for problems with: • Many variables • Large domains • Based on domain reduction paradigm • Quantitative constraints • Integer domain • Reduction by arithmetic relations • Greater than (>) • Less than (<) • Equals (=)
Drawbacks of quantitative constraints • Too many (trivial) solutions that differ by: • 1 pixel position, or • 1 milliseconds in timing • Not sufficiently expressive e.g. cannot specify “no overlap” constraint • Too low level e.g. A.X2 B.X1
Solution: qualitative constraints • Example “A not overlap B”, “B after C” • Advantages: • More intuitive • More expressive • Smaller domains • Qualitative solutions translate automatically to lower level quantitative constraints • New problem: What if constraints are insoluble?
Solution: Constraint Logic Programming • Combine Prolog unification and backtracking with constraint solving • Use Prolog rules to generate constraints • Backtrack when constraints are insoluble
Cuypers generation engine • Multiple layers: • Prolog rules to generate constraints • Qualitative constraints translate to quantitative constraints • Solution of both constraints provides sufficient information for final presentation
Cuypers demo: scenario • Client:User is interested in Rembrandt and wants to know about about the “chiaroscuro” technique • Server: Query database • Server: Generate constraints according to: • System profile • User profile • Network profile • Server: Solve constraints / revise constraints • Server: Generate SMIL presentation • Client: Play presentation
Conclusions • Quantitative constraintsare insufficient for automatic multimediapresentation generation. Also need • Qualitative constraintsto allow intuitive and effectivehigh level specification, and • Backtrackingfor revising specific constraintsthat cause the entire set to fail
Future directions • Best-first instead of depth-first • Choose “best” among possible solutions • Needs evaluation criteria • Improve knowledge management • Make design knowledge declarative and explicit • Preserve metadata in final presentation • Use standardized and reusable profiles
Need to make trade-offs • Semantics • Convey message • Aesthetics • Clear / nice layout • Resources • Screen size, bandwidth • Dimension may result in conflicting goals
Quantitative Constraints % csp(+Ids, -Boxes) csp([IdA,IdB],[box(IdA,[x1:AX1, …]), box(IdB,[x1:BX1,…])]) :- % get values maxX(MaxX), maxY(MaxY), height(IdA,HeightA), widtht(IdA,WidthA), … % define domains [AX1,AX2,BX1,BX2]::[0..MaxX], [AY1,AY2,BY1,BY2]::[0..MaxY], % set width & height AX2 – AX1 #= WidthA, AY2 – AY1 #= HeightA, … % constraints AX2 #< BX1, % A left-of B AY1 #= BY1, % A top-align B, … true.
Multimedia and Constraints • Constraint Logic Programming • Domain reduction • Backtracking • Unification (matching rules) • Qualitative Constraints • Non-integer domain • Allen’s 13 temporal interval relations in three dimensions
Qualitative Constraints • Example: • Two images, A,B • A left or right of B • A not above or below B
Qualitative Constraints % csp(+Ids, -Graph) csp([IdA, Idb], [edge(IdA,IdB,x,NoOverlap),…]) :- % define domains NoOverlap :: [b,b-,m,m-], Overlap :: [d,d-,s,s-,f,f-,e], % constraints edge(IdA,IdB,x,NoOverlap), % B not-overlap A edge(IdA,IdB,y,Overlap), % B overlap A true.
Qualitative Constraints • Reasoning • Inverse: edge(A,B,D,Value) <=> inverse(Value,RValue),edge(B,A,D,RValue). • Equality edge(A,B,D,V1), edge(A,B,D,V2) => V1 #= V2 • Transitive edge(A,B,D,VAB), edge(B,C,D,VBC) => tr(VAB,VBC,VAC), % rule generation algorithm edge(A,C,D,VAC). • Translation rules to quantitative domain edge(A,B,D,b) => node(A,D/2,V2), node(B,D/1,V1) V1 #< V2.
Problems in generating multimedia • Text documents are flexible • Add page, scrollbar, • Template models • Wrap text around images • Multimedia documents are less flexible • No pages or scrollbars, no line-breaking or hyphenation • Not based on text-flow • Feedback needed • Linear process model does not work for multimedia
Quantitative Constraints • Example: • Two images, A,B • A left-of B • A top-align B
Cuypers generation engine • Rhetoric/Semantic • Sequence, Example • Communicative devices • Bookshelf, Slideshow • Qualitative Constraints • A before B • Quantitative Constraints • A.X2 < B.X1 • Presentation • SMIL