Big Data and Usability

1. Big Data and Usability H. V. Jagadish Univ. of Michigan

2. Scaling • Computing systems can be made to scale. • Human capacity does not scale • Scaling is used to address Big Data. • So solutions only address the computing part and ignore the human.

3. Ex.: Graduate Student Appl. • If you have 5 applications, as a human expert you can form a pretty good judgment. • If you have 5000 applications, what will you do?

4. How Big is Big Data? • Standard is “too big to handle using current techniques” • Doesn’t have to be Petabytes.

5. Possible Solution • Have machine replace human. • Can be made to work in some domains. • But need human intelligence in many others. • Have to think about how to get machines to support humans better.

6. Ex.: Fast Browsing • The Skimmer system [SIGMOD 2012] can be used to browse faster. • Works by adaptively down-sampling rows depending on user’s scroll speed.

7. Not All Vs Are Equal • Big Data is defined as having: • Volume • Velocity • Variety • (Lack of) Veracity • … • But everyone immediately latches on to size.

8. Variety >> Volume • Volume is very easy to measure. • Companies (and even system research groups) can easily boast about their accomplishments in handling big volumes of data. • Variety has no accepted metric. • It is possibly a much bigger challenge than size. • Yet gets very little attention.

9. Where Does Variety Arise? • Across data sets. • Need to integrate many disparate sources of information. • Within one data set. • Ex.: Customers are all different • May be ignored at great peril!