1 / 61

Not More than You Can Chew Bite-sized tactics to make sense of your metrics #13NTCbite

Not More than You Can Chew Bite-sized tactics to make sense of your metrics #13NTCbite. Jo Miles Food & Water Watch @ josmiles. The Problem. Sometimes working with data feels like biting off more than you can chew. The Solution.

kiora
Download Presentation

Not More than You Can Chew Bite-sized tactics to make sense of your metrics #13NTCbite

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Not More than You Can Chew Bite-sized tactics to make sense of your metrics #13NTCbite Jo Miles Food & Water Watch @josmiles

  2. The Problem Sometimes working with data feels like biting off more than you can chew

  3. The Solution Break your great big data needs into delicious, bite-sized pieces.

  4. Today, we’ll talk about: • The right way to think about data • Data analysis, without a lot of math • Working more efficiently with data • Making time for data in your busy day • Using your data for better decision-making • How to be a Data Analysis Honey Badger!

  5. We will NOT talk about: • Technical ways to collect your data • What metrics to collect • Statistics • Data modeling • Advanced analysis software

  6. 1. Why analyze?

  7. Data should be useful

  8. NTEN/Idealware Report: "State of Nonprofit Data” It says a majority of orgs don't find their data useful for decision-making. And many have trouble tracking it.

  9. Challenges in using data effectively • Data collection/quality • Expertise • Technology • Prioritization/time

  10. Challenges in using data effectively • Data collection/quality • Expertise • Technology • Prioritization/time

  11. What does it take to be a data analysis honey badger? Not as much as you’d think… Wikimedia: MatejBatha

  12. 2. Set yourself up for success

  13. Only track data that's useful • If it’s not useful… Don't spend time on it! • Ask yourself: • Is this data likely to change much over time? • Will I go back and look at it? • What will I do differently if this data looks good or bad?

  14. Know what to track • Every system is different • Every organization is different Track this stuff!

  15. Track on a timeframe that's useful to you Daily, weekly, monthly, quarterly, yearly…

  16. Measure regularly • Create an easy format to track metrics: • Email response rates • Action rates • Donation counts • Update it diligently • In 30 minutes or less • Schedule reports if you can • Summarize with charts • Look for trends and surprises

  17. A monthly report example

  18. Analyze as needed • Measuring won’t tell you everything • Use analysis to answer specific questions… • …but only when you need to.

  19. Analysis = Big Questions • Are our activists more likely to donate? • What sources give us the most engaged supporters? • Who should we target for a second gift? To become sustainers? • Who is unsubscribing, when, and why?

  20. Analysis = Exploration When you see something strange, dig in: • Who? • What? • When? • Where? …and look for evidence of why. Flickr: DG Jones

  21. More on digging in • Compare: • Across issues • Across categories • Across timeframes • Look for trends over time • Segment by: • Donor status • Activist status • Time on list

  22. Learn Excel! • Sort and filter • Charts • Formulas • Pivot tables Yes, seriously!

  23. Remember:Some bites are too big to chew • If you can't get good data... • Or if it really is too much work... • Sometimes it’s okay to with your gut! Flickr: Roger Smith

  24. Building a data-driven culture • When making decisions, ask: • Is there data that can help us? • Are we making big assumptions that data could prove? • Do what the data says – or have a good reason not to!

  25. Building a data-driven culture • Sharing is caring! • Share both your data and your results! • Better yet, show others how you draw insight from your data Flickr: 1225design

  26. 3. Developing your number sense

  27. You don't need statistics

  28. Get cozy with numbers • Listen to the stories they tell you • Know how to • Compare them • Manipulate them • Explain them

  29. What’s “normal”? • It looks something like this: • Without using statistics: • Get familiar with your typical ranges • Learn to eyeball what “normal” is These are “normal” values, at a glance

  30. Benchmark against yourself • Benchmark reports tell how you're doing compared to others. • That can be useful. • But others aren't you!

  31. Do-it-yourself benchmarking (email performance) • Report on ALL your emails from past year: • # delivered • # of opens • # of clicks • # of actions/donations (if possible) • # of unsubscribes Flickr: Samantha Chapnick

  32. Do-it-yourself benchmarking (email performance) • Calculate average rates for all messages. E.g. for average open rate: • Take sum of all opens • Divide by sum of all delivered This is your benchmark open rate. • Do the same for other metrics.

  33. Do-it-yourself benchmarking (email performance) • For bonus points, group messages by type: • Advocacy • Fundraising • National • Local • Etc. ...and calculate those benchmarks rates.

  34. Using your do-it-yourself benchmarks • Compare each new message against benchmark rates. • Update periodically with recent data. • Watch how benchmarks change over time.

  35. 4. Lies, damned lies… …and the lying liars who tell them

  36. Lying with data:The path to the Dark Side “Come to the Dark Side. We have cookies.” Flickr: Antony Hell

  37. Don't compare apples to oranges Compare-able Not compare-able! Flickr: b1ue5ky

  38. Don't compare apples to oranges What you can do: Compare things that are as similar as you can make them. Remove oddballs.

  39. Don't make too much of a small difference What you can do: Calculate lift. Pay attention to significant differences. Ignore the others.

  40. Calculating lift • "Lift" shows how different two rates are. • Rate X1 is your baseline. How much better/worse is rate X2? • Positive means X2 is better. • Look for lifts of at least 5%. • Anything over 10% is pretty good.

  41. Graphs can lie, too What Excel actually gave me by default: But the lift is only 2%!

  42. What’s happening here?Should we panic? What you can do:Check your graphs. Are they clear? Are they truthful?

  43. Don't draw conclusions from tiny numbers • Rule of thumb: to know anything about your data, you need: • At least 100 data points. • Preferably 500 data points. • More is better. What you can do: Say "we think this might be a trend, but we need more data."

  44. This has a name: Statistical Significance • Yes, it’s statistics… • But it's important! • Look for online tools that calculate this for you

  45. Don't draw careless conclusions Remember: correlation does not imply causation. xkcd.com

  46. Don't draw careless conclusions • Data tells you WHAT, not WHY • Have you heard this one? • "80% of our activists have taken action on food issues. • Therefore, they must like food issues best, so we should send more food alerts." What you can do: Always give caveats when guessing the "why" behind the numbers. Second-guess your assumptions.

  47. Don’t lie… even to yourself It’s so tempting to lie with your data! You must resist the temptation. Don’t go to the dark side!

  48. 6. Sharing with decision makers

  49. First step: What’s your point? • You are presenting numbers for a reason. What reason? • Share your conclusions first. • THEN share the supporting data Isn’t it obvious what’s happening here? 7 Don’t let your numbers speak for themselves!

  50. Who is your audience? • Some people “get” numbers • Others do not • Present your data in the way that speaks to them ?

More Related