Deco — Declarative Crowdsourcing

1. Deco — Declarative Crowdsourcing Scoop — The Stanford – Santa Cruz Project for Cooperative Computing with Algorithms, Data, and People Hector Garcia-Molina, Aditya Parameswaran, Hyunjung Park, Alkis Polyzotis, Jennifer Widom Stanford and UCSC

2. The Big Picture Same as everyone else’s… Declarative queries DBMS like thing Web

3. The Big Picture Same as everyone else’s… Declarative queries WSQ/DSQ [Goldman & Widom, SIGMOD 2000] DBMS like thing Web

4. Primary Focus (distinguishing features?) • Theoretical foundations • Generality / flexibility • Query optimization

5. The Deco Data Model Goals for a new data model (shamelessly stolen from myself) • Well-defined • Understandable • Sufficiently expressive (and not more) • Similar to existing models • Implementable

6. The Deco Data Model Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

7. Rest of This Presentation 1. Small motivating example 2. “Relations and other stuff” 3. Mapping from conceptual schema to actual schema 4. Semantics: “valid instance” (actual to conceptual) Not included in presentation • Enough examples, especially data • Normal forms (BCNF/4NF) • Metadata/annotations • Various details… Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

8. Small Example 1. Fetch 2. Resolve 3. Join User view ⋈ resolution rule o resolution rule Anchor fetch rule Chez Panisse Bytes French fetch rule Dependent Dependent fetch rule fetch rule

9. Relations and other stuff • Relations • Attributes designated as anchor or dependent • Resolution rules— dealing with uncertainty • Fetch rules— “access methods” to externally obtained data Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

10. Relations and Attributes R (restaurant, address, rating, cuisine) S (address, city, zip) Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

11. Relations and Attributes R (restaurant, address, [rating], [cuisine]) S (address, [city, zip]) [dependent attribute groups] Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

12. Relations and Attributes R (restaurant, address, [rating], [cuisine]) S (address, [city, zip]) [dependent attribute groups] anchor attributes Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

13. Resolution Rules R (restaurant, address, [rating], [cuisine]) S (address, [city, zip]) One resolution rule per dependent attribute-group restaurant,address rating (F=avg) restaurant cuisine (F=dup-elim) address city,zip (F=majority) LHS  RHS with (black-box) function F Given LHS values and one or more RHS values, F returns zero or more (new) values for RHS Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

14. Fetch Rules R (restaurant, address, [rating], [cuisine]) S (address, [city, zip]) LHS  RHS with procedure P Given LHS value, procedure P can obtain RHS values from external source(s) restaurant,address rating restaurant cuisine address city,zip rating  restaurant,address cuisine  restaurant,address rating,cuisine  restaurant,address  restaurant,address Schema designer End user relations and other stuff relations   anchor attributes (subset of) anchor  dependent group(s) dependent group(s)  anchor Conceptual schema automatic (system) Actual schema RDBMS

15. Conceptual Schema (end user) R (restaurant, address, rating, cuisine) S (address, city, zip) Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

16. Actual Schema R (restaurant, address, [rating], [cuisine]) S (address, [city, zip]) restaurant,address rating restaurant cuisine address city,zip One RR per dependent attribute-group “Deco tables” [decomposed] • One for anchor attributes • One for each resolution rule A1(restaurant, address) A2(restaurant, address, rating) A3(restaurant, cuisine) A4(address) A5(address, city, zip) Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

17. Actual Schema R (restaurant, address, [rating], [cuisine]) S (address, [city, zip]) restaurant,address rating restaurant cuisine address city,zip One RR per dependent attribute-group • Fetch Rules ― add tuples • Resolution Rules ― resolve uncertainty (for query result) A1(restaurant, address) A2(restaurant, address, rating) A3(restaurant, cuisine) A4(address) A5(address, city, zip) Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema + RDBMS

18. Valid Instance of Database • Given: • Current contents of Deco tables (actual schema) • Fetch Rules and Resolution Rules • Valid instance (conceptual schema) is any state of relations obtained by: 1. Fetch— add tuples to Deco tables by invoking FR procedures 2. Resolve— resolve dependent attributes using RR functions 3. Join — full outerjoin of Deco tables for each relation Schema designer End user relations and other stuff relations Conceptual schema automatic (system) Actual schema RDBMS

19. Small Example 1. Fetch 2. Resolve 3. Join User view ⋈ resolution rule o resolution rule Anchor fetch rule Chez Panisse Bytes French fetch rule Dependent Dependent fetch rule fetch rule