The FileSender project: collaborative opportunities in NREN land

1. The FileSender project:collaborative opportunitiesin NREN land Guido Aben, AARNetPty Ltd APAN33, Chiang Mai

2. Talk purpose: • Share our experience with grass-roots, collaborative inter-NREN software/service development • Tell you about the availability of a great service that you can run at your NREN for free, without any serious effort • Open the door wide for interested contributors (code, translations, testing, funds…)

3. FileSender, very high level

5. FileSender: User Adoption Statistics from AARNet’s FileSender installation: • <1 helpdesk call per week (mostly re: federation!) • ~ 99.995% uptime • ~4% of users sending >2GB files, without instruction (seen 67GB in the field) • At February 2010, >2000 users& >6 TB uploaded stats are world-viewable at https://cloudstor.aarnet.edu.au/mrtg/

6. FileSender: Existing Installations ASU Uni Lithuania AARNet Australia ARNES Slovenia CESNET* Czech Rep. BELNET Belgium FCCN Portugal HEAnet Ireland i2CAT Catalonia Internet2* USA Okinawa Institute of Science & Technology*Japan NIIF* Hungary North-West UniSouth Africa RESTENA* Luxembourg SIDN* Netherlands SRCE Croatia SURFnet* Netherlands TERENA* Netherlands UNINETT Norway • 18 known installations across 16 countries: * test installation

7. FileSender: project factsheet • Started April 2009 • 1.0 released 31 January 2011 • Core team of 5 NREN people in 3 countries • Agile, online collaboration • Back: php/apache/postgresSQL/simpleSAMLphp • Front: Flash + HTML5 (for >2GB) • Version 1.5 (beta out yesterday!): http://blog.filesender.org/2012/02/13/filesender-1-5-beta1-released/ plugin-free for >2GB uploads; MySQL support • Handles hanzi/hiragana/katakana/quốcngữ (W3C, i18n, UTF8; auto language selection support)

8. Splendid isolation: top-down SW dev “Every good work of software starts by scratching a developer's personal itch” – Eric S. Raymond • Match personal itch with internal req’ment • Run survey;look for clues to back up new service idea • Propose swiss army knife to funding body • Start in-house development straight away, using toolchain most convenient to developers • Run resultant SW on server box of your taste • Package what you have on the deadline; declare success*; move on. • Patch bugs with documentation *at 100 users, at 200 logins, at 95% uptime, take your pick

9. How we optimised for collaboration • Still started with a personal itch of course! (who would dare contradict Eric Raymond?) • But then consulted with int’l eR / NREN orgs, exchanged ideas, inspected similar services and SW • Reduced scope of initial planned release • Turned our dev group outward, not inward • Planned for ease of install, thus wide deployment, thus critical mass and sustainability • Resisted urge to use comfortable/hip toolchain; • Tested, retested and tested again: refused to fix with documentation; instead fixed user experience

10. Old vs. New The start’s the same – a personal itch. It’s very hard to start from requirements taken form surveys or focus groups. You have to trust your instincts somehow. “It's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them.” – Steve Jobs

11. Old: Run survey;look for clues to back up new service idea New: consulted with int’l eR / NREN orgs, exchanged ideas, inspected similar services and SW (even tried to get 3rd party code public-domained; weren’t successful, but would try again) Our challenges are unlikely to be unique.And our survey reach is limited.Increasing the catchment area for “accidental discoveries” helped. Surveying the same people again doesn’t, usually, for us. If your challenge is really unique, perhaps you’re just scratching your own itch? Should that be funded with infrastructure monies?

12. Old: propose swiss army knife to funding body New: Reduced scope of initial planned release A tradition exists of promising the world on a silver platter to the funding bodies and then having to push lots of features overboard during the project. This makes planning hard and distracts from the real outcome. Instead we chose to work on a pared-down initial release that we could pay for without funding. Lay the foundations first, don’t get distracted by artificial deadlines. If the resulting product is great, we’ll run a funding application later.

13. Old: start in-house development straight away, using toolchain most convenient to developersRun resultant SW on server box of your taste(script, install binary dependencies, chroot and recompile at will) New: turned our dev group outward, not inward - the full “open source” works: license, bug tracker, project wiki, packaging, releasing Planned for ease of install everywhere, thus wide deployment, thus critical mass and sustainability Resisted urge to use comfortable/hip toolchain(decided against java; picked php, forbade ourselves to use external dependencies) We made a conscious choice to sacrifice coding speed for long-term viability.The project was designed as an open source project; to have to be open about internal disagreement was… a learning curve. To have to refrain from grabbing a “more elegant” tool or language was hard at times. Having to wait for consensus… frustratingThe result has been a much bigger userbase, more input, more corner case testing and a much reduced per-user cost

14. Old: Package what you have on the deadline; declare success*; move on. Corollary: run service as-is, treat SW as abandonware, patch bugs with documentation *at 100 users, at 200 logins, at 98% uptime, take your pick New: tested, retested and tested again: refused to fix with documentation; instead fixed user experience

15. We like it so far It’s been a rewarding journey: we have the service we wanted our project looks like it’s sustainable We’re keen to try more of this: • More features delivered for filesender in a modular, collaborative way • A new service idea delivered in collaboration Thoughts? Volunteers?

16. More Info: www.filesender.org -- main project site blog.filesender.org -- news, releases And finally for those of you who’ve really become enthusiastic about contributing: www.assembla.com/spaces/file_sender/wiki/Workplan_2012