i-Care Project Dietary-Aware Dining Table: Observing Dietary Behavior over Tabletop Surface

1. i-Care ProjectDietary-Aware Dining Table: Observing Dietary Behavior over Tabletop Surface Keng-hao Chang, Shih-yen Liu, Toung Lin, Hao Chu, Jane Hsu, Polly Huang, (Cheryl Chen) i-space Laboratory National Taiwan University

2. What is it? • A dietary-tracker built into an everyday dining table • Track what & how much you eat over tabletop surface • Motivation • We are what we eat • Food choices affect long-term & short-term health • Show a demo video

3. Smart Everyday Object • Digital-enhanced everyday objects • Provide digital services • Support natural human interactions • Natural human interactions = inputs to digital services • Goals • Providing digital services without (users) operating digital devices → better usability • Human-centric computing: technology adapting to users rather than users adapting & learning about technology

4. Outline for Reminder of Talk • Related work • Approach • Assumptions & Limitations • Design & Implementation • Experimental Evaluation • Future work

5. Related Work • Dietary trackers • Shopping receipt scanner (GaTech) • Chewing Sound (ETH) • My food phone (startup) • Intelligent surfaces • Load sensing table (Lancester) • Smart floor (GaTech, NTU) • Posture Chair (MIT) • What’s new here? • Accuracy • Fine-grained tracking • Simultaneous concurrent interactions

6. Contribution claims • It is a fine-granularity (automated) dietary tracker. • It can track multiple concurrent interactions from multiple individuals over the same tabletop surface. • People usually don’t eat alone • It is an enhanced loading sensing table.

7. General Approach • RFID tags on food containers • Two sensor surfaces on table • Each surface is made of cells • RFID reader surface • Detect RFID(s) in each cell • Weighting surface (load cells) • Measure weight change in each cell • Track the food path from container(s) → container(s) →mouth using these two sensor surfaces

8. Assumptions (Limitations) • Closed system rather than open system. • Food transfers among tabletop objects and mouths, no external objects and food sources • Users identified by personal containers (personal plates and cups) • Food containers tagged with RFID tags • No cross-cell objects • No leaning their hands on the table • Not a mobile tracker

9. Single Interaction Example • Bob pours tea from the tea pot to his personal cup, and drinks it • Detect tea transfer from one container to another container • Identify the presence & absence of containers • RFID tags on containers • tag-food mapping • Track tea transfer • Weight change detection • Weight matching algorithm

10. Single Interaction Example • Bob pours tea from the tea pot to personal cup, and drinks it • Put on tea pot. • RFID tag appears • Weight increases ∆w3 • Pour tea! • |∆w3 - ∆w1 | ≈ ∆w2 • Pick up tea pot. • RFID tag disappears • Weight decreases ∆w1 • Pour tea? • Weight increases ∆w2.

11. Single Interaction Example • Bob pours tea from the tea pot to personal cup, and drinks it • Put on cup. • RFID tag appears. • Weight increases ∆w2. • Drink tea? (only if no match) • Amount | ∆w2 - ∆w1 | • Pick up cup. • RFID tag disappears. • Weight decreases ∆w1.

12. Concurrent Interactions Example • Bob pours tea & Alice cuts cake • Cut cake • Weight decreases ∆w2 • Pour tea? • Cut cake? • Weight change ∆w • Pour tea • Weight increases ∆w1

13. Concurrent Interactions Example • Multiple, concurrent person-object interactions • The larger the cell, the higher the possibility of concurrent interactions over a cell • Cell size = average size of container • Reduce the possibility of concurrent interactions over one cell

14. Design Architecture Applications (Dietary-aware Dining Table) Dietary Behaviors Behavior Inference Engine Tag-object mappings Intermediate Events Event Interpreter Common sense semantics Sensor Events Weight Change Detector Object Presence Detector Weighing surface (weighing sensors) RFID Surface (readers)

15. Inference Rule

16. Experimental setup • 2 Dining settings • Afternoon tea • Chinese-style dinner • 2 Parameters • # of participants • Predefined vs. Random Sequence A Keng-hao Willy

17. Experimental Results

18. Afternoon Tea (Single User) cut a piece of cake and transfer it to the personal plate; pour tea from the tea pot to the personal cup; add milk to the personal cup from the creamer; eat the piece of cake from the personal plate; drink tea from the personal cup; add sugar to the personal cup from the sugar jar. Afternoon Tea (Multi-users) A cuts cake and transfers it to A’s personal plate; B pours tea from the tea pot to B’s personal cup; A pours tea to A’s personal cup while B cuts a piece of cake and transfers it to B’s personal plate; A adds sugar from the sugar jar to A’s personal cup while B adds milk from the creamer to B’s personal up; A eats cake and B drinks tea; B eats cake from B’s personal plate while A drinks tea from A’s personal cup; A pours tea from the tea pot to both A’s and B’s personal cups. Predefined Activity Sequence

19. Activity Recognition Accuracy in Scenario #3

20. Causes of Misses in Scenario #3

21. Activity Recognition Accuracy in Scenario #4

22. Causes of Misses in Scenario #4

23. Conclusion • It is a smart object and a smart surface • It supports natural user interface • It supports fine-grained dietary tracking at individual level • It is about human-centric computing • Accuracy can be improved further

24. Future Work • Improving recognition accuracy • Removing constraints (assumptions) • Persuasive computing • Encourage balanced diet • Encourage proper amount of diet

25. Questions & Answers Thank You