Implementing a Data Analytics Solution using Agile Scrum

1. Implementing a Data Analytics Solution using Agile Scrum

2. Consulting and Software Done Right AgileThought is a national system integratorand custom software firm, staffed by the best professionals in the business. Our mission is to fundamentally transform the way people, organizations and companies accomplish their work and build deliver software products. For over 14 years, AgileThought has been a trusted partner to Fortune 1000 companies who have benefited from our services. Whether you need a custom software solution, expert guidance on cloud and mobile strategies or you're looking to improve team efficiency and workflow, we can help.

3. Jose Chinchilla, MCSEData Analytics Leadjose.chinchilla@agilethought.comwww.linkedin.com/in/josechinchilla @sqljoe

4. Agenda Overview of a Data Analytics Solution Typical features and deliverables Popular platforms Common pitfalls Overview of the Agile Scrum approach Why Agile Scrum? Backlog Roles & Ceremonies Metrics Bringing it all together: Data Analytics + Agile Scrum workshop

5. Data Analytics Data analytics is the pursuit of extracting meaning from raw data using specialized computer systems. These systems transform, organize, and model the data to draw conclusions and identify patterns.  - Informatica

7. Data Analytics History Reports Decision Support Systems Business Intelligence Data Warehouse / Data Mart Data Mining Big Data Data Science (ML/AI) • Descriptive · Predictive · Cognitive · Augmented

8. Typical Features & Deliverables

9. ETL / ELT

10. Data Model Physical Data Model • Tables and Columns • Data types • Primary and foreign keys • Database naming standards • Measures and Dimensions • Data formatting • Relationships • Business naming standards Logical Data Model

11. Reports • Canned · Ad-hoc · Subscription • Filtering • Slicing & Dicing • Drill-down • Drill-through • Cross-platform • Contextual • Responsive

12. Dashboards • KPIs: Actuals vs Target • Status cards • Trend lines • Time Intelligence Year-over-Year Year-to-Date Previous Year Month-to-Date 3-Month Average

13. Tools & Platforms Data Analytics

14. Tools & Platforms

15. Popular Platforms Microsoft SAP Oracle IBM Tableau Qlik MicroStrategy Domo Tibco

16. Microsoft Tools & Platforms Power BI Excel SQL Server Data Platform Integration Services Analysis Services Reporting Services Azure Data & Analytics Data Factory SQL DB SQL DW Data Lake Analytics ML Spark

17. Common Pitfalls Data Analytics

18. Common Pitfalls • Assume that you have good data • Lose fidelity and lineage of data • Publish reports with visualizations your average user does not understand • Forget about color blind users • Neglect to realign the solution with changing business processes

19. Agile Scrum Software development framework based on iterative development, where requirements and solutions evolve through collaboration.

20. A Project Analogy How the customer explained it How the sales executive described it How it was architected What the requirements say How users perceived the end product What the customer really needed What the programmer wrote How it was fixed

21. Test/Fix Deploy Value Realization : “Waterfall” Value Realization Month 4 Month 1 Month 3 Month 2

22. Value Realization : “Agile” Value Realization Month 4 Month 1 Value Realization Value Realization Value Realization Month 3 Month 2

24. Scaled Agile Delivery: Roles & Responsibilities

25. Sprint Ceremonies

26. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Scrum Meetings • Sprint Review • Sprint Retrospective Backlog Refinement Main driver: Product Owner Attended by: Entire Team Categorized as part of Themes, Epics, Features Story prioritization for upcoming sprint Story size estimation (Fibonacci sequence) Refine stories and clarify acceptance criteria User Story format: “As a [persona], I want to [do something] so that I can [some reason]”

27. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Scrum Meetings • Sprint Review • Sprint Retrospective User Story: Format Story Title: Story Description: As a [who] I want to [what] so that I can [why] Assigned To: Estimate:

28. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Scrum Meetings • Sprint Review • Sprint Retrospective User Story: Example Story Title: Load a table with a summary of sales data Story Description: As a Sales Manager I want to analyze sales data so that I can track sales over time by Customer and Product. Assigned To: Jane B. Story Size: 2

29. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Scrum Meeting • Sprint Review • Sprint Retrospective Sprint Planning Main driver: Delivery Lead with Product Owner Attended by: Entire Team Sprint goals alignment Team capacity and availability Tasks creation and effort hours Story lead assignment Scheduling of group discussions Scrum board setup 2-3 week sprint length

30. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Scrum Meetings • Sprint Review • Sprint Retrospective Tasks Story Title: Load a table with a summary of sales data Task Description: Extract sales data from the Sales Detail table Assigned To: Mary C. Estimate: 6 hours

31. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Stand-up Meetings • Sprint Review • Sprint Retrospective Daily Stand-up Meeting Main driver: Delivery Lead or Scrum Master Attended by: Development Team 15-20 minute meeting with Scrum board Synchronize team’s work Each team member must answer 3 questions: What did I worked on yesterday? What will I be working today? Are there impediments blocking my work today?

32. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Stand-up Meetings • Sprint Review • Sprint Retrospective Scrum board

34. Jira Trello

35. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Stand-up Meetings • Sprint Review • Sprint Retrospective Sprint Review Main driver: Team member or Delivery Lead Attended by: Entire Team Main audience: Product Owner Main goal: showcase / demo work that was completed

36. Scrum Ceremonies: • Backlog Refinement • Sprint Planning • Daily Stand-up Meetings • Sprint Review • Sprint Retrospective Sprint Retrospective Main driver: Delivery Lead or Scrum Master Attended by: Development Team Main audience: Team Main goal: team self-assessment / lessons learned Three main questions: What worked well for us? What did not work well us? What ca we improve going forward?

37. Agile Scrum Metrics Sprint burndown Net Promoter Score

38. 12 Guiding Principles of Agile Highest priority is customer satisfaction Embrace changing requirements Deliver working software frequently Daily collaboration Motivated team members Personal interactions are most efficient A final working product is the ultimate measure of progress Agile promotes sustainable development Technical excellence and good design enhances agility Simplicity is an essential element Self-organizing teams produce best architectures, requirements, and designs Regular team self-assessment and correction promotes effectiveness https://agilemanifesto.org/

39. Common Pitfalls Poorly formulated stories Unclear acceptance criteria Stories too large (13+ story points) Underestimating story size and complexity Tasks that take more than a work day (6+hours) Not accounting for team members’ time off Not respecting the team members’ sprint Turning daily stand-ups in discussion meetings Not allowing for architectural spikes Being too rigid to changes in business priorities

40. Data Analytics+Agile Scrum

41. Data Analytics + Agile Scrum • Agile Scrum allows Data Analytic team members to: • Focus on a data source / data set at a time • Get immersed in business processes and business rules • Understand the big picture while breaking deliverables in smaller chunks • Get more frequent feedback from end users • Agile Scrum allows users to: • Better understand complexities and oddities of the data • Express their needs with more detail as the solution is being delivered • Explore the data and drive insights with more focused use cases

42. Data Analytics + Agile Scrum • What works? • Architectural spikes allow developers to understand the data • Frequent delivery and feedback makes for a better end product • Sketches • Focusing on MVP initially • Cross-functional teams • What doesn’t work as well? • Lengthy requirements • Lengthy business documentation • Siloed teams

43. Group Exercise Let’s build an analytical solution!

44. Reference Architecture Source 1: Sales Database ETL Data Warehouse Power BI Reports Source 2: Excel Spreadsheet with Product Categories

45. Analytical Solution: Source Relationships

46. Analytical Solution: Destination Table

47. Creating a Prioritized Backlog Exercise # 1

48. Sprint 1 Deliverable Who is this deliverable for? Sales Manager What is the ask? Sales, Customer and Product data from SQL data source merged with Product Categories from Excel data source into a summary table in the Data Warehouse that gets updated every month. What is the purpose of the deliverable? He wants to be able to analyze data over time and summarize it by Customer and Product Categories.

49. Exercise (10 mins) • Write 3 to 5 stories (each story in a sticky note) Add acceptance criteria in the back Assign priority Assign estimated story size Assign a lead developer • Create 2-3 tasks per story each in a sticky note Assign a developer responsible for the task Assign hours Story Title: Story Description: As a [who] I want to [what] so that I can [why] Assigned To: Estimate:

50. Running a Stand-up Meeting Exercise # 2