User-Based Innovation & Communities Drive Commercial Systems Software

1. User-Based Innovation & Communities Drive Commercial Systems Software James Hamilton GM SQL Server WebData Development & SQL Security Architect http://research.microsoft.com/~JamesRH JamesRH@Microsoft.com 2004.04.15

2. Introduction • I’m an unrepentant commercial software guy :-) • Have done some non-commercial S/W work in spare time • Ported g++, gdb, and Taylor UUCP to AIX 1.3 • Ran a UUCP site for years • System Software Focused • 11 years at IBM • Ada & C++ Development Manager • Lead Architect DB2 UDB Database • 7 years at Microsoft • Windows2000 Base Development • SQL Server development team in various roles • Relational System Development Manager • Security Architect • General Manager WebData Development • Interested in better understanding & harnessing user community contribution to S/W systems What follows are my observations & opinions and do not necessarily represent a Microsoft position

3. Agenda • Driving innovation: users or manufacturer? • Revolutionary change often not user driven • Examples from DB & TP world with which I’m most familiar • Redundant Array of Inexpensive Disks (RAID) • Relational database • Users prime drivers of product evolution • Community drives commercial S/W success • Commercial S/W has always been community dependent • Why not go open source & gain the community contribution? • Parallels between commercial & non-commercial s/w efforts

4. Revolutionary change: often not user driven • The fundamental changes are often not user driven • Users do drive incremental & evolutionary change • Some examples from DB world: • Online index create • Online re-org • XML datatype • Automated multi-system administration • … • Revolutionary change examples from systems world: • RAID & Relational DB “…Well-managed companies that have their competitive antennae up, listen astutely to their customers, invest aggressively in new technologies, and yet still lose market dominance”

5. Redundant Array of Inexpensive Disks • Disks were expensive, IBM dominated, & growing at sub-Moores law rates • Commodity disk much less reliable & 1/5 to 1/10 the capacity of enterprise disk • “A Case for Redundant Arrays of Inexpensive Disks (RAID)”– Patterson, Gibson, Katz • Base observation: Commodity disks with redundancy can be combined to produce larger & more reliable storage SIGMOD, June 1988

6. EMC Symmetrix DMX3000 • 84 TB Storage capacity • 576 3 ½” commodity Seagate (usually) SCSI Disks • 256 GB Memory & 100 1 Ghz PowerPC CPUs • 10x to 15x storage cost premium over commodity disk • EMC annual revenue: $6.24B (2003 10K filing) • RAID overall annual revenue: $13B (1998 Disk/Trend Report)

7. Relational Database • “A Relational Model of Data for Large Shared Data Banks”– E. F. Codd CACM 13,6 (June 1970) • “Some industrial motivation and then straight to the math”– Irv Traiger • Broadly debated: CODASYL vs Relational • SIGFIDET and SIGMOD conferences • First IBM System R relational DB user: • MIT Sloan School of Management (System R Phase 0 proto 1975) • System R code base later became IBM SQL/DS • later DB2 for VM & VSE — product still available • Manufacturer research rather than user community pull Relational DB Theory: E.F. Codd SQL Language: Don Chamberlin

8. Relational Database Market • Overall annual Relational DB market: $8B • Gartner expects 86% of DB revenue to be Relational by 2005 • Many DB & TP industry innovations driven by user partnerships • IBM TPF: American Airlines SABER • Project started 1959 • IBM IMS: Rockwell-NASA Apollo Program • Released 1969 • IBM CICS: Partnership with Public Utilities • PUCICS released 1968 • The innovation required user involvement but they didn’t drive it Source: Gartner 2001

9. Agenda • Driving innovation: users or manufacturer? • Revolutionary change often not user driven • Examples from DB & TP world with which I’m most familiar • Redundant Array of Inexpensive Disks (RAID) • Relational database • Users prime drivers of product evolution • Community drives commercial S/W success • Commercial S/W has always been community dependent • Why not go open source & gain the community contribution? • Parallels between commercial & non-commercial s/w efforts?

10. Communities Drive Commercial S/W • Product Support • User community support is the best way to provide scalable, available, high quality product support • It’s difficult to invest enough in a dedicated support team to replace a community based program • Examples form engineering team in which I work: • Participation required of all team members • Newsgroups, customer presentations, feedback sessions, service team internships, customer requirements DB, work with MVP community, sample programs, … • Marketing and Sales also heavily community driven • ISV & reseller sales model common • Shareware & community sales sites

11. Community Example:www.boatdiesel.com

12. Why not use Community Development? • S/W business driven by important, difficult to reproduce S/W aggregations • Separates lack-luster profit from the truly impressive • S/W business cost of entry is very close to zero • Profit of most entrants unexciting • Large S/W systems with critical mass very valuable: • SAP, Windows, Oracle, DB2, … • Large S/W aggregations are actively protected • But they are open in many ways • Most support interfaces to allow 3rd party extension • Source is typically available in a controlled way • Open-source S/W systems also actively protected • However, branches are possible and sometimes succeed • Commercial S/W open at key interfaces but user source code mods typically not supported • Both commercial and non-commercial S/W systems dependent upon community for success

13. Summary • Most product innovation is driven by users • Improvements in speeds, feeds, & features • Applications of existing technology to new domains • Revolutionary changes often not user driven • Fundamental new approaches to existing problems • Especially approaches that are: • Not backward compatible • Don’t adequately solve the entire breadth of the problem domain • All products, whether commercial or not, depend upon user community for support & innovation • User driven support only affordable & effective option • User Community source of most innovation • User development community typically the driver of even non-open source product success

14. Microsoft