Rowing Motion Capture System

1. Rowing Motion Capture System Simon Fothergill Ph.D. student, Digital Technology Group, Computer Laboratory Jesus College graduate conference May 2009

2. Overview • The Bigger Picture • Previous work • Problem • Process • Data Capture System • Results • Future work

3. The Bigger Picture • Sentient Computing! • Computer Vision • Pattern Recognition & Machine learning • A long way to go!

4. The Bigger Picture – Watching Humans • Physical Performances • Heath care • What are they doing? • How well are they doing it? • How should be improved? • How should they be told?

5. Previous Work - Activity / Gesture recognition • Motion capture methods have included: • Blob tracking • Point trajectories • Recognition techniques have included: • Single frame • Multiple frame • Parametric

6. Learn the quality of a performance from body part trajectories • Minimise markers using redundancy • Complex trajectories, continuous score • Flexible rubrics require learning • Different types of expert labelling: • Explanations • Non-specific / specific • Different granularities of quality • Which sections of the trajectory are how relevant? • One section of a can depend on many aspects

7. Process Extract and select features Learning Trajectories Features Capture motion Performance Capture video Video Inference model Expert coach labels with their judgement Learn Judging Performance Features Judgement Extract and select features Capture motion Evaluate Trajectories Inference model

8. Data Capture System ECS Erg Control Power Motion sensitive LED markers Wii controllers

9. Data Capture System - Architecture TCP/IP Fire wire Video camera (30Hz) PC PC Java / C client C server Wii library Bluetooth library Bluetooth Bluetooth Nintendo Wii controller Nintendo Wii controller IR 1024x768 camera (100Hz) IR 1024x768 camera (100Hz) C server Buffer Wii controller Wii controller

10. Data Capture System – Calibration and operation Server Client Batch Storage 4 x 2D coordinates Stereo calibration Calibrate WMCS Calibration Erg calibration Calibrate labeller Label markers Triangulation Update ECS if necessary ECS Transform to ECS Live operation 4 x 3D coordinates Detect strokes Calculate stats Save picture Control camera Display on GUI Log data Log files Encodes video

11. Data Capture System • Example video

12. Preliminary Results • Preliminary results have been obtained using a dataset of 6 rowers and the complete trajectory of the erg handle only. Binary classification over stroke quality was done using tempo-spatial features of the trajectory and a neural network. Two training methods were compared. Classification accuracy across given number of performers, for quality of individual aspects of technique.

13. Summary and Further Work • Data capture system and how it fits into the bigger picture • More information is available on the feature extraction & selection and inference algorithms. • A larger data set would allow conclusive results to be obtained • Feature extraction and selection methods that address using the relevant segments of the relevant trajectories • More sophisticated modelling based on particle filters • Supports multiple body parts and labelling methods • Uses a distribution of motion vectors to probabilistically track the “quality so far” as the stroke evolves.

14. In Conclusion • Advertisement! • Acknowledgements • Professor Andy Hopper, Dr Sean Holden, Dr Robert Harle • Members of the DTG and Rainbow groups, Computer Laboratory • Jesus College, JCBC and the Graduate society • References • Optical tracking using commodity hardware, Hay, S.; Newman, J.; Harle, R.; ISMAR 2008. Page(s):159 - 160 Thank you! Questions? Please come down to the boathouse and use the data capture system!