Mobile development with less complexity, more confidence, and fewer manual steps

1. Mobile development with less complexity, more confidence, and fewer manual steps Benjamin van der Veen Partner, Survant James @bvanderveen PNSQC, Oct 17-19 2016 SOFTWAREQUALITYCONFERENCE PACIFIC NW PNSQC ™

2. Background • Started doing mobile with the 1st-gen iPhone in 2008 • But before that… • web development since 2000 • a lot of Flash and Windows desktop apps • low-level, high-performance networking/IO experience • I can attest that mobile is really hard!

3. Mobile is hard • Harder than web apps and backend services • Lots of state and state transitions • End product has to look good and be correct • The practice is far less mature than web development • Automated testing? • Automated deployment?

4. The climate • Lower barriers to entry • More engineers, more libraries, more choices, more dependencies, more problems • More horsepower available on more devices • Fewer engineering constraints, more problems • More devices, more problems • More designers • More problems

5. The climate • Automated testing and deployment still uncommon

6. What to do? • Write, maintain, and run automated tests. • Write, maintain, and run a manual test plan. • Allow the platform to guide you known-good, low-complexity UX and UI solutions. • Automate deployment as much as possible, and deploy frequently.

7. Automated tests • What are you supposed to test in mobile apps?

8. Automated tests • Web app testing • Database mutations (input) • Database queries (output) • Form/model validations (error cases) • Smoke/integration/happy path tests (state transitions)

9. Automated tests • Mobile apps don’t often have… • Forms • Databases • Direct responsibility for querying structured data

10. Automated tests • But they do have… • Complex rendering logic • Highly state-dependent user flows • Dependencies on system services (GPS, push notifications, calendar, contacts, etc)

11. Automated tests • DON’T: try to write automated tests around rendering or layout • Very hard to write automated tests for user interface behavior, e.g.: • Text or images are laid out correctly in all cases • When the user taps a button it turns a color • When X scrolls Y appears or disappears • &c

12. Automated tests • DO: test logic that can be separated from the UI • Create a ‘backend’ module • Everything that can be decoupled from the UI goes in • It’s tested if and only if it’s in the backend module • Mock out system services (push, contacts, location, network, etc) and test for • User approves a permission • User denies a permission • User has no contacts • Location can’t be found • No network • &c

13. Automated tests • State machines • Model the kinds of input events the app can receive • Model the kinds of output and side-effects the app can have • Write a formal state machine around those models, and exercise it in an automated fashion

14. Automated tests • State machine example • Given the following input events: • The app has started • The “app has started before” flag not set • Assert the following side-effects: • The ‘first run’ screen is displayed • The “app has started before” flag is set

15. Automated tests: recap • Create a backend module • Everything in the backend has an automated test • Everything with an automated test is in the backend • Agnostic of UI • Mock out system services • Use state machines • Don’t write automated tests for UI

16. Manual testing • Test “everything else”! • Layout and rendering code • Text or images are laid out correctly • When the user taps a button it turns a different color • When X scrolls Y appears or disappears • etc • Onboarding • Integration with e.g., web services • Meets real-world requirements e.g., location services • Adequate performance

17. Manual testing • Keep the testing plan in your source repo along with the app and the automated tests • Update it in step with the codebase • Make it part of code review

18. Manual testing • Ideally, run it exhaustively… • On each feature branch before merge into master • On master after a feature merge • Before releases • Try to strike a balance, because it can potentially get time consuming • Although if you keep your use-cases simple, maybe not… • Use your judgement. Maybe… • just run relevant parts before merging • just run stuff you know changed since the previous release

19. Recap • Mobile is hard • Write, run, and maintain automated tests • Write, run, and maintain manual tests

20. Allow the platform to guide you • Okay strap in, because we are about to go to 6 G’s.

21. Allow the platform to guide you • MVP (minimum viable product): • “The simplest possible product that allows you to validate your market hypothesis and allows you to learn enough to iterate the product forward.” – Wikipedia (?)

22. Allow the platform to guide you • Use the built-in controls and navigation paradigms the platform gave you! • Don’t fight them! • Don’t try to improve upon them! • Don’t innovate here!

23. Allow the platform to guide you • Designers can very easily violate this principle without knowing it.

24. Allow the platform to guide you • Example #1: • The difference between a button and a button with a custom background and ‘depress’ states can be 1 minute and 1 line of code vs 4 person-hours and 20 lines of code.

25. Allow the platform to guide you • Example #2: • On iOS, the difference between • a “just-in-time” login screen that slides up from the bottom and back down when dismissed, and • a “just-in-time” login screen that slides in the from right and out to the left • …is that the former is trivial to implement and the latter is extremely awkward (maybe impossible) to implement.

26. Allow the platform to guide you • These are really subtle differences!

27. Allow the platform to guide you • The process that results in the design of the product must be aware of the constraints imposed by the manufacturing process.

28. Allow the platform to guide you • What happens when the design goes to developer?

29. Allow the platform to guide you • Mode 1 (“Pitch it over the fence.”) • Designer designs a feature. • Developer spends (a perhaps unexpectedly long) time on the feature. • Developer may or may not achieve designer’s vision in that time • If the vision is achieved: • Great! You got lucky. • If the vision is not achieved: • Even more time is required (rinse, repeat), or— • Vision is abandoned • Time has been wasted? People frustrated? Trust frayed? • Deadlines or market opportunities missed?

30. Allow the platform to guide you • Mode 2 (“Collaboration between design and development”) • The developer recognizes that, relative to a similar alternative, a given design decision… • creates a large amount of code complexity • takes significantly more time to implement • results in code that’s difficult to maintain or change • If developer offers feedback to designer • Design is changed to reduce implementation complexity, savings achieved. (This is where we want to end up!) • If the developer does not offer feedback to the designer • See Mode 1 (“Pitch it over the fence”)

31. Allow the platform to guide you • Cultivate a feedback loop between design and development! • Does the team have liberty (trust?) to just do the simple thing? • Is team empowered (trusted?) to collaborate on reducing complexity? • Is the team incentivized to do so? • Or are they billing hourly? ;) • Do design and development have • open lines of communication? • a collaborative rapport? • shared product goals? • alignment on how time and money is best spent?

32. Allow the platform to guide you • It can get hairy! • Live and die by MVP • Focus on high-level user flows and business value • “Pixel perfect” is dangerous, expensive, and elusive • “Flow perfect” is achievable, expedient, and extremely valuable • Design should never be a one way flow from designer to developer! • Or we aren’t collaborating! Collaboration is essential for expedient, efficient, sustainable product development

33. Recap • Mobile is hard • It ain’t like web apps • Write automated tests where it makes sense • Use a manual test plan for verifying visuals and integration with real-world services • Allow the platform to guide the team to expedient solutions

34. INTERMISSION

35. UI code • Why is this so hard!?

36. UI code • It amounts to configuration. The only way to see if it’s correct is to see if it’s correct.

37. UI code • UI code rarely encodes logic in the usual sense of the word

38. UI code • Expensive to modify — write once, read never!

39. UI code • Bottom line: it’s hard to meaningfully test in an automated fashion. • Don’t innovate • (too soon) • Don’t fight The System • (unless you’re well-funded and have political connections on the inside)

40. UI code • Componentize where possible • Work with design to create a reusable palette of widgetry • Avoid new, ad-hoc design solutions on each new screen • Collaborate and iterate!

41. UI code • Get designers and developers in the trench together • Get flow-perfect, not pixel-perfect • Focus on business value

42. Recap • Mobile is hard • Test what you can, visually inspect the rest • UI code is hard to reason about • Don’t fight The System • Focus on biz value

43. Deployment • Compare web apps — it’s all figured out! • Mobile apps — not so much • Code signing • :( • Tedious • Lots of manual steps • The dreaded review process • Impediment to deployment

44. Deployment • Just script everything. Start banging on it. • Building the app • Building the tests • Running the tests • Building a fully-baked, ready to deploy package • Code signing • Asset generation • Icons • Screenshots • Use CI to exercise all of the above regularly

45. Deployment • Speaking of CI. Don’t outsource CI. • There are a million also-ran CI-as-a-service startups out there and in our experience they are all: • Slow • Finicky • Break builds by changing their environments without warning • Are not appreciably different from each other • Are not appreciably different from running Jenkins or Hudson on a Mac Mini in your closet • Except that they periodically break for no apparent reason • …and anyway you probably don’t want your signing keys on hosted CI

46. Deployment • …which leads us to: write your own everything. • Own and maintain your tools • Don’t get tangled up with some huge graph of open source dependencies • Your needs are unique! Invest in your own ergonomics.

47. Deployment • Always be in the App Store review queue • Lately, Apple’s turnaround times are 2-3 days (for us?) • Iteration 0: submit a build to Apple • Iteration 1: make a few changes, cut a release, submit it as soon as Iteration 0 is approved. • Iteration N: since you always have a good build in your CI pipeline, just submit iteration N every time iteration N-1 is approved by Apple.

48. Bottom line • Test coverage, automated and manual • Minimal UI complexity • Less code • reusable code • Collaboration and feedback between design and development, design is kept ‘in check’ by platform realities • Automation • Build and automated testing • Deployment — build, signing, asset generation • CI — all paths tested frequently

49. Results • Confidence! • You minimized complexity where it hurts the most—UI • And you’re running manual QA on that • For everything else, you’ve got automated tests • Ability to emit builds at the drop of a hat • Internal/beta users can always play with the latest • Minimal developer headache associated with creating a build • More predictable and stable release cadence

50. The End Questions?