MPEG-4 Toward Solid Representation

1. MPEG-4 Toward Solid Representation Alain Mignot and Pierre Garneau IEEE Trans. on Circuits and Systems for Video Tech., Vol. 14, NO. 7, JULY 2004, pp. 967-974. Presented by: Reza Aghaee Multimedia Course(CMPT820) Simon Fraser University March.2005

2. Agenda • Introduction • Future of 3D Standards • Solid Representation: MPEG-4’s Answer to 3D Challenge • Implications of Rendering Mechanisms • Impacts on Applications • Conclusion

3. Introduction • Traditional 3D Image Creation • Tessellation • Models are created of individual objects using link points that are made into a number of individual polygons • Geometry • Polygons are transformed in various ways and lighting effects are applied • Rendering • Transformed images are rendered into objects with very fine details

4. Introduction (cntd.) • Performance is the number of polygons processed per second. • MPEG-4 is introducing a new and different approach. • This method can be used in various 3D applications. (Games/CAD/CAM/CAE)

5. Future of 3D Standards • 3D Technology is at a Turning Point • 10 years old OpenGL is reaching its limits. • It’s confirmed by user-programmable parts of the pipeline. • This limited programming capability, in the form of vertex and pixel shaders, appears to be the easiest solution.

6. Future of 3D Standards • Shader Programs May Not Be the Solution • It is actually a step backward, in the opposite direction of the beneficial standardization process. • Graphic APIs display 3D scenes on almost any device. • Drivers take care of hardware differences other than performance.

7. Future of 3D Standards • Shader Programs May Not Be the Solution (cntd.) • A software driver can not simulate most shader programs. • Developers should provide different versions for different graphic boards. • Shader Programs do not bring any additional information to the rendering process concerning the scene itself.

8. Future of 3D Standards • Historical Considerations • Higher performance was often achieved by including parts of pipeline into dedicated hardware. • First, later stages of the pipeline were performed in hardware and front portion in software driver. • 3D technology should extend its domain on functionalities performed by application.

9. Future of 3D Standards • Limitations of the 3D Pipeline Common to OpenGL and DirectX • Algorithms designed to create the illusion of depth and continuity. • 3D rendering is a set of techniques that have nothing to do with geometric objects. • In real 3D space, objects may influence each other even if invisible.

10. Future of 3D Standards • Limitations of the 3D Pipeline Common to OpenGL and DirectX (cntd.) • Two parallel data structures needed • One for describing the scene ( objects with geometric attributes) • One for the set of polygons (each object in its position relative to observer) • First structure for managing interactions • Second structure only for rendering

11. Future of 3D Standards Traditional 3D Game Production

12. Future of 3D Standards Traditional CAD rendering

13. Projected Evolution of 3D standards • Data structure of geometric objects and their attributes is common in most applications. • Most applications share a common framework based on geometrical and physical properties of objects. • By including this framework in the rendering engine the whole process is well improved.

14. Solid Representation in MPEG-4 • MPEG-4 introduces the purely mathematical definition of shapes. • These shapes are based on algebraic shapes combined with arithmetic shapes. • These shapes are completely independent of rendering process. • These functionalities are referred to as “Solid Representation”.

15. Solid Representation in MPEG-4 • In MPEG-4, solid representation functionalities deal with: • Reducing the size of files to be transmitted or shared by transmitting constructive commands rather than results. • Increasing the geometrical precision of rendered objects by managing a polynomial representation of volumes. • Manipulating complex objects combined by solid operations.

16. Solid Representation in MPEG-4 • Solid Primitives • Any solid object has a 3-D form defined by a skin, which delimits the inside from the outside. • If equation of surface is unknown the volume will be divided into simpler pieces until they have known forms. • These primitives will then be assembled in order to constitute the original complex shape. • Figure 3 is a complex shape made of a set of solid primitives. Figure 3

17. Solid Representation in MPEG-4 • There are two approaches to define a surface: • A Parametric Equation giving spatial coordinates of each surface point. • Implicit Equation of algebraic surfaces. • Implicit Equation of a Sphere • (Px – Cx ) 2 + ( Py – Cy ) 2 + (Pz - Cz) 2 - R2 = 0 • Eq=0 means point is on the surface,for Eq<0 point is inside and for Eq>0 point is outside the volume.

18. Solid Representation in MPEG-4 • A quadratic equation (second-degree equation) allows to define the entire quadrics family: • An equation of the fourth degree allows to define the quartics family:

19. Solid Representation in MPEG-4 • Implicit second-degree equation defining quadrics is:where for each point coordinates (X0…X3) the result is:F(X0,…,X3) < 0 ; F(X0,…,X3) = 0 ; F(X0,…,X3) > 0whether the point is inside, on the surface or outside.

20. Solid Representation in MPEG-4 • The point coordinates are made homogeneous by adding X3 . • A surface of fourth-degree will have 35 coefficients. • Unbound surfaces should be bounded to be processed and displayed. • Cylinder equation is an example of unbound volumes.

21. Solid Representation in MPEG-4 • In MPEG-4 the coefficients of the quadric (second-degree implicit surface) may be defined by six geometric control points. P0,P1 : 2 contact points on the quadric. P2,P3 : 2 poles of the construction tetrahedron. P4,P5 : 2 passing points of the quadric. • Each point is defined using homogeneouscoordinates allowing the point to be sent to infinity (affine geometry)

22. Solid Representation in MPEG-4 • In MPEG-4 two Geometry nodes implement algebraic surfaces: • Implicit Node: Defines the surface by the coefficients of the polynomial. • Quadric Node: Defines the surface by the six control points explained in the previous slide.

23. Solid Representation in MPEG-4 • “Arithmetic of Forms” is a logical modeling system for solid objects. • Description of a solid object takes the form of a solid tree made up of operators and operands. • Operands are primitives or more complex solid objects. • Operators are mostly union, intersection and logical subtraction.

24. Solid Representation in MPEG-4 • Each basic geometric primitive splits space into three regions – external, boundary, and internal – coded by integers 0, 1, 2. • This ternary coding of space tells us the density of every point in space. Constructive Solid Geometry (CGS) tree and corresponding model 3D

25. Solid Representation in MPEG-4 • There are three basic sets of operators • General Arithmetic Operators on Densities: • For instance, addition, multiplication, and difference of densities of two volumes. • Below is multiplication of two forms F0 and F1.

26. Solid Representation in MPEG-4 • Three basic Sets of Operators (cntd.) • Arithmetic Operators with Ternary Logic: • They use ternary logic only. • Examples are union and intersection. • Below is ternary intersection of F0 and F1.

27. Solid Representation in MPEG-4 • Three basic Sets of Operators (cntd.) • Densities Filtering • In MPEG-4 a set of test functions is applied on the root of the solid tree to filter densities while keeping the filtering inside the tree. • Examples can be “Equality Filter” (F0==F1) Results of Solid Operations

28. Implications of Rendering Mechanisms • If the volumes must be displayed very precisely some techniques can be used to render the output. • If one accepts less geometric precision, it is possible to consider tessellation of implicit surfaces before or after applying solid operation and rendering.

29. Implications of Rendering Mechanisms • The functionalities of solid representation in MPEG-4 including Implicit, Quadric and SoldRep Nodes are completely independent from the rendering method. • All the solid operators are independent from the rendering process too.

30. Impacts on Applications • Compactness • Suitable for sophisticated web applications including online gaming on small-constrained devices mobile phones and PDAs. • Complete model of a historical castle with a level of detail from roof frame to door openers takes 50 Kb.

31. Impacts on Applications The following example is a model solely based on SolidRep geometry. Details of the Leihorra villa’s model Complete Solid Model of Leihorra villa

32. Impacts on Applications • Exact Geometry • The exact geometry is preserved up to the decoder. • It allows very precise scientific applications as CAD/CAM or simulations. • Below is the simulation of the Canadian Space arm from Canadian Space Agency.

33. Impacts on Applications • Embedded Topology and 3D Properties • There is a wrap-up of the topology and 3D solid properties (e.g. constituent matter and physical properties) • Local manipulation, exploration of the model and very accurate collision detection is allowed. • The picture is an inside view of the villa as the result of a solid Operation.

34. Impacts on Applications • There are many games claiming that the player can destroy almost any object to find other hidden spaces. • This model can help implement these models without very heavy programming techniques. Model cut with a laser beam Complete Original Model

35. Conclusion • Polygonal pipelines can’t present the future of 3D standards. • Solid representation can transfer very complex models in their most exact geometry and in a very compact way. • Even without a dedicated hardware current CPUs are powerful enough to provide real-time processing. • Solid representation is now a part of the MPEG-4 part-16. • New APIs will be developed including object description to accelerate solid representation.