Coding of 3D Mesh Sequences Nikolce Stefanoski ITG-Fachtagung 3.2 Hannover 2006

1. Coding of 3D Mesh Sequences Nikolce Stefanoski ITG-Fachtagung 3.2 Hannover 2006

2. Overview

3. Overview • Dynamic meshes • a sequence of static meshes • constant connectivity (temporally consistent, encode once) • geometry changes throughout the mesh sequence Frame: 001 280 380

4. Outline • Single-resolution coding of 3D mesh sequences • Spatially scalable coding of 3D mesh sequences • layer design • coder

5. Predictive compression scheme Vertex locations are encoded in a predefined order using the predictive coding paradigm: Frame f = 1 2 3 4 5 …

6. Order of encoding of vertex locations • Vertex traversal algorithm defines the order of compression of vertex locations within a frame • Vertex traversal is connectivity guided • region growing breath-first approach Legend: blue: not yet visited vertices gray: already visited vertices

7. Vertices are traversed in a frame to frame fashion. Following the predictive coding paradigm each new vertex location is predicted from already encoded vertex locations of the previous and the current frame: residual original location predicted location Vertex Traversal Algorithm: Encoding • Quantized residuals are encoded

8. Vertices are traversed in the same order as used for encoding. Vertex locations are reconstructed to: reconstructedvertex location reconstructed residual predicted location Vertex Traversal Algorithm: Decoding quantization error:

9. Quantization • Quantization is performed in each space direction uniformly • Quantization bin size Δ is calculated relatively to the longest bounding box edge max Δ = max/2q

10. q=11 q=10 q=9 q=8 q=7 q=6 Quantization The effect of quantization:

11. In this framework different realizations for the predictor are analyzed concerning their compression performance: • Extended Lorenzo predictor (ELP) • Motion vector averaging (mvavg) • Angle preserving predictor (angle) Vertex Traversal Algorithm: Predictors General assumption: neighboring vertices perform similar motion

12. In this framework different realizations for the predictor are analyzed concerning their compression performance: Frame f Frame f-1 Vertex Traversal Algorithm: Predictors Extended Lorenzo Predictor (ELP) • predict motion vector for vertex location using parallelogram prediction (blue dashed line) • linear predictor • perfect predictor when only translation is performed from f-1 to f

13. In this framework different realizations for the predictor are analyzed concerning their compression performance: Vertex Traversal Algorithm: Predictors Motion-Vector Averaging Predictor (mvavg) • predict motion vector for vertex location (blue solid vector) through motion vector averaging • linear predictor • perfect predictor when only translation is performed from f-1 to f Frame f Frame f-1

14. In this framework different realizations for the predictor are analyzed concerning their compression performance: Vertex Traversal Algorithm: Predictors Angle Preserving Predictor (angle) • preserves the angle between the two triangles in frame f-1 while prediction in frame f • non-linear predictor • perfect predictor when rigid motion is performed from frame f-1 to f Frame f Frame f-1

15. Comparison of predictors • a vertex-wise L2 norm (da) is used for evaluation • significant compression gains of the angle preserving predictor against linear predictors in the area of bit-rates over 7 bpvf • combined (angle+mvavg) predictor shows best performance • realized by frame-wise selection between two predictors • motion vector averaging reduces the influence of coarse quantization errors to the bit-rate

16. Comparison of compression algorithms (Wavelet) (PCA) (Dynapack) In all 3 papers the same vertex-wise L2 norm (da) is used for evaluation.

17. Comparison of compression algorithms High quality mesh sequences: • significant gains in bit-rate • gains of over 25% in the area of errors below 0.03 (≥12 bit quantization) Low quality mesh sequences: • dominant influence of quantization errors to bit-rates • compression performance rapidly drops in the area of bit-rates below 4 bpvf

18. Outline • Single-resolution coding of 3D mesh sequences • Spatially scalable coding of 3D mesh sequences • layer design • coder

19. Layer design • Layers = meshes with different resolution (no. of vertices) • Layers should be consistent in temporal direction, i.e. frames part of the same layer should have same connectivity Layer K: Layer K-1:

20. Our approach of layer design: Connectivity based simplification Important observation: vertices have valence 6 in average (no. of neighboring vertices) valence dispersion around 6 in usually low (few high valence vertices) Layer design

21. Layer design Valence distribution:

22. Layer design • Goal: preserve low valence dispersion while doing simplification • remove patch-wise only vertices with valence ≤ 6 • remesh patches in a manner that minimizes valence dispersion around 6

23. Outline • Single-resolution coding of 3D mesh sequences • Spatially scalable coding of 3D mesh sequences • layer design • coder

24. Spatially scalable compression Illustration of frame dependencies:

25. Prediction 434 067 943 [01101….] layer K-1 vertex locations layer K vertex locations vertex location to be predicted Space-time prediction Geometry Encoding • Quantization of the prediction error • Encoding Arithmetic Coder 8-12 bit/coordinate Motion vector averaging Frame: f-1 f

26. Results

27. Results

28. Thank you !