Creating Big Data Success with the Collaboration of Business and IT Teams

1. Creating Big Data Success with the Collaboration of Business and IT Teams By Edward Chenard

2. - Started big data at Best Buy, was working in big data at GE before it was called big data. • - Set up one of the first hadoop clusters in Retail and the Midwest. • - Won tax innovation credits for my work on big data • - Tekne finalist for big data innovation • - Set up big data, data science and data visualization teams • Managed teams as large as 300 with product portfolios of over $4B • I spend my time in cold places Edward Chenard edward@echenard.com Twitter: Echenard Slideshare: Echenard

3. Everyone is Jumping on to Big Data

4. The Reality of Big Data • As many as 3/4 of big data projects fail according to one Gartner study. • The third is that 39 percent of the failure of Big Data project is attributed to the fact the data is siloed and there’s not a lot of cooperation in gaining access to that data. Now that is the oldest problem in the history of IT. - Infochimps • 1. They focus on technology rather than business opportunities. • 2. They are unable to provide data access to subject matter experts. • 3. They fail to achieve enterprise adoption. Terradata's top three reasons why big data projects fail. • Lack of alignment. Business and IT groups are not aligned on the business problem they need to solve but instead are tackling it from a technology perspective. Lack of true commitment from business stakeholders also makes alignment harder to achieve. Peter Sheldon - Forrester Analyst

5. Big Data at Most Companies IT Business Big Data

6. How a typical big data project takes place • Someone hears about big data and then seeks funding. • Other teams want to own it. Months of fighting takes place over ownership. • The opportunity is either lost or the mission of the project gets altered. • Teams work in silos, poor communication takes place as teams spend more time playing CYA. Achievement: Project failure with cost over runs, deadlines missed and lack of focus.

7. No One Team can Handle Big Data Alone

8. Current state of big data collaboration

9. What does Big Data Really Mean to Business “The ability to take data - to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it's going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for elementary school kids, for high school kids, for college kids. Because now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it.” Hal Varian

10. Everything is about discovery

11. Why a focus on collaboration? • Projects fail for simple reasons, lack of understanding the need for better collaboration and then knowing how to implement that collaboration, helps to ensure success. • Failure does not need to be an option • Big data is the future of how we live and work, but only if we get it right. Big data can be bigger than ecommerce in terms of impact on how we live.

12. Everyone Discovers

13. Data Discoverers “The Data Discoverers looks a lot like you and me, but what’s different is their preoccupation with personal data. They are relentlessly digital, they obsessively record everything about their personal lives, and they think that data is sexy. In fact, the bigger the data, the sexier it becomes. Their lives - from a data perspective, at least - are perfectly groomed.” data as a lifestyle

14. Data Discoverers Data Discoverers are setting the trend in what will be common place in just a few short years. More people will want to use their data and the consumerization of data and technology will continue. As this trend goes, only organization that learn to merge the various disciplines of strategy, analytics and IT, will be successful data as a lifestyle

15. How We Need to Look at Discovery

16. Discovery is the leading emerging interaction category of the Age of Insight

17. Complex ecosystems: multi-channel experiences everyware environments Service models dynamic perspectives Reactive data

18. Activity Centered Thinking

19. How Different Functions See the Same Issue “Understand the quality performance of a system so I can better determine if I need to replace it.” - IT “Understand a portfolio's exposures to assess portfolio-level investment mix.” - Strategy Manager “I need to understand the customer trends in the data so I can better create models.” - Analyst

20. Identifying Modes “Understand the quality performance of a system so I can better determine if I need to replace it.” - IT “Understand a portfolio's exposures to assess portfolio-level investment mix.” - Strategy Manager “Understand the customer trends in the data so I can better create models.” - Analyst Mode = ‘Comprehend’ (understand)

21. Comprehending ‘To generate insight by understanding the nature or meaning of an item or data set’ e.g. “I need to analyze and understand consumer-customer-market trends to inform brand strategy & communications plan” – Director, Brand Image Each Team has the same goal, to understand, what they may want to understand is often different but not exclusive or limit to the other team’s need to understand.

22. Identifying Modes “I needvisibilityinto the systems my colleagues are using in order to maximize the network ROI for the company.” - IT “I need to identify customers/marketers/dealers failing & at risk of de-branding based on performance problems.” - Strategy “I need to identify the best customer/consumer/region targets for our brand/products.” - Analyst Mode = ‘Explore’

23. Modes are the verbs of discovery scenarios.

24. 9 distinct modes Locate Verify Monitor Compare Comprehend Explore Analyze Evaluate Synthesize

25. Where to Start

26. The Business Value Framework Initiatives Customer Acceptance Business value Customer Acceptance Business value Focus on Customers Focus on Internals Customer’s Wallet Share Perceived Value Ease of Data Collection Ease of Implementation Ease of Data Collection Ease of Implementation Value Perceived Customer’s Wallet Share Pre-recorded Production Flexibility Different Products Customer Needs More Products Production Efficiency Automated and prompted Timeliness

27. How work gets structured Clearly articulated vision for personalization and recommendations, precisely defined goals with how to measure. Defined scope of the product. Market strategy, customer segmentation, prioritization, org focus, measurement and incentive systems Production process, flexibility at scale, efficiency, relationship management, benchmarking, metrics, initiatives

28. Framing Collaboration Value (Shared): Show me the money!?! • Measurable Results • Multi-Channel Case Studies Strategy: Where are you headed? IT: What Tools and Why • Buy vs. Build • Open source options • Alignment with Analytical Infrastructure • Speed to Market • Privacy Considerations Big Data Collaboration • MapReduce, Hadoop • Cassandra, The Cloud • Pig, Hive, • HDFS Analyst: Who, How, Where? • Data Scientist vs. Statistician • Where to find talent? • Retain, Train • Offshore vs. Onshore • University involvement

29. Always Remember: Data, Insights, Actions

30. Collaboration helps to achieve where others fail.

31. Thank You! • Edward Chenard • Twitter: Echenard • Email: edward@echenard.com • Blog: CrossChannelPrairie.com