The Business Case for Agile Languages

1. The Business CaseforAgile Languages Stephen Ferg Bureau of Labor Statistics ferg_s@bls.gov www.ferg.org

2. In the beginning... • System programming languages • Assembler, C • Fortran, Cobol, Algol • PL/1, Pascal, Basic, C++, Java • Command languages • JCL • TSO CLists, CMS "execs" • Batch files

3. "Little Languages" appear... • awk • sed

4. UNIX shell scripting Rexx Tcl, TK Perl Python PHP Ruby Scripting languages evolve Some have higher-level object-oriented features that make them powerful application development languages in their own right.

5. Agile programming languages "Scripting" languages aren't just for scripting any more. We really should stop calling them "scripting" languages. "Agile" languages would be more accurate.

6. What is an agile language? • Interpreted • requires a run-time interpreter or virtual machine • Untyped or dynamically typed • No data declarations • No compilation step

7. The hallmark of agile languages... Vastly increased productivity!

8. Productivity ::= the number of machine instructions that a programmer can produce per year. A programmer can write roughly the same number of lines of code per year regardless of language. "Scripting: Higher Level Programmingfor the 21st Century" by John K. Ousterhout IEEE Computer magazine, March 1998 -- http://home.pacbell.net/ouster/scripting.html • Assembly Language • One line = one machine instruction • System programming languages • One line = 3 to 7 machine instructions • Scripting/agile languages • One line = 100s to 1000s of instructions

9. Language Levels and Productivity 1000 Scripting Languages VB 100 Python, Perl, Ruby, TCL Instructions/Statement Java 10 C++ C System Languages Assembler 1 None Degree of Typing Strong From "Scripting: Higher Level Programming for the 21st Century" by John K. Ousterhout. This version prepared by Dana Moore and updated by Stephen Ferg

10. The results indicate that, for the given programming problem, 'scripting languages' ( Perl, Python, Rexx, Tcl ) are more productive than conventional languages." University of Karlsruhe (Germany) Technical Report, March 2000

11. A good example... Python Developed in 1991 by Guido van Rossum • Best of breed • Powerful / flexible • Easy-to-learn / use • Open source A fan of Monty Python's Flying Circus

12. Developer Reports • I find that I'm able to program about three times faster in Python than I could in Java, and three times faster in Java than I could in C. • Andy Hertzfeld • When a 20,000 line project went to approximately 3,000 lines overnight, and came out being more flexible and robust ... I realized I was on to something really good. • -- Matthew "Glyph" Lefkowitz • ...the lines of Python code were 10% of the equivalent C++ code. • -- Greg Stein, eShop

13. Productivity = Web speed May 4, 2004 When burglars brought down the Internet link to Ziff-Davis' Manhattan offices, open-source software - and Sean Gallagher's personal Web server - kept eWEEK.com's stories flowing. For about six hours yesterday, the staff of eWEEK.com was treated to a lesson in emergency improvisation. But thanks to 123 lines of Python code, running on a $7-a-month personal Web site on an Apache Web server somewhere in California, you probably didn't notice.

14. Fast to write... but maintainable? ... the real punchline of the story is this: weeks and months after writing my Python program, I could still read the code and grok what it was doing without serious mental effort. • Eric S. Raymond, author of The Cathedral and the Bazaar

15. "Programming is fun again!" • Over and over on comp.lang.python : "Now that I've discovered Python, I enjoy programming again!"

16. The ability of agile languages to increase productivity and maintainability leads to...increasing use of scripting/agile languages.

17. Scripting languages represent a different set of tradeoffs than system programming languages. They give up execution speed and strong typing but provide significantly higher programmer productivity and software reuse. This tradeoff makes increasing sense as computers become faster and cheaper compared to programmers. John Ousterhout, developer of Tcl I think there is a trend ... Dynamically typed languages such as Python, Ruby, and even Smalltalk will be mainstream industrial languages in the coming years. Robert C. Martin, author of Agile Software Development

18. Sounds good, but ...

19. Is open-source safe ?

20. Can you trustopen-source software? Python is an "open-source" language. It has no vendor. Does that mean we'll have support problems? What about... • Vendor longevity? • Consulting & training support? • Books and reference materials? • Tools? IDEs, debuggers, screen-painters?

21. Is anybody using open-source software?Is open-source software used in the Federal government?

22. Linux Apache MySql PHP | Perl | Python Who is using open-source software? "LAMP" Apache has overwhelmingly dominated the Web server market since 1996. PHP is the most popular Apache module, running on almost 10 million domains (over a million IP addresses). ... and then there's ...

23. DoD In 2002, a Mitre study found 115 FOSS (free and open-source) products in use in the U.S. Dept. of Defense. • http://egovos.org/pdf/dodfoss.pdf

24. ... and IBM • In September 2003, IBM began promoting Linux with a series of television ads depicting a young boy receiving lessons from famous innovators and teachers. The boy represents the next generation of humanity, learning from teachers who – like the open-source community – freely share their accumulated expertise.

25. Why would a Federal agencyuse open-source software? FromGovernment Computer News November 20, 2000 ... The experience of the NASA Acquisition Internet Service (NAIS) development team ...

26. Our proprietary Web development application that promised interoperability with another vendor’s database software failed to interoperate... Then we discovered Perl and have been using it for the last five years for all NAIS applications. Recently, price restructuring for a commercial DBMS threatened to consume most of the NAIS budget. We decided to convert to MySQL. Our tests showed MySQL could perform NAIS functions faster. Cost of the optional technical support was about 1 percent of that for the commercial product. Technical support for MySQL has been excellent when we needed it, plus there are hundreds of Web sites that offer free help and support for such open-source products. We plan to evaluate the Apache HTTP Server to correct limitations of the commercial Web server we currently use.

27. Who is using Python? What are they doing with it? • Industrial Light & Magic, maker of the Star Wars films, uses Python extensively in the computer graphics production process. • Disney Feature Length Animation uses Python for its animation production applications.

28. Google, a leading internet search engine, is powered by Python. • Yahoo uses Python for its groups site, and in its Inktomi search engine. • The Philips (formerly IBM) Fishkill semiconductor manufacturing facility uses Linux and Python for factory tool control.

29. NASA uses Python in several large projects, including a CAD/CAM system and a graphical workflow modeler used in planning space shuttle missions. • The National Institutes of Healthand Case Western Reserve University are building cutting-edge genetic analysis software with Python. • The National Weather Service uses Python to prepare weather forecasts.

30. Lawrence Livermore National Laboratories is basing a new numerical engineering environment on Python. • The Theoretical Physics Division at Los Alamos National Laboratory uses Python to control large-scale simulations on massively parallel supercomputers.

31. US Navy uses Python & Zope for a web based workflow system • US Dept. of Agriculture uses Python & Zope for massive collaboration • US Dept. of Labor uses Python & Zope for the Workforce Connections learning management system

32. ... or "What about vendor longevity?" What if Guido got run over by a bus?

33. Common Misconceptions • Open-source software has no vendor • The vendor of an open-source product is the (single) person who developed it. • Anybody can change (and maybe mess up) an open-source product.

34. The Python Software Foundation • A non-profit organization • Holds Python's intellectual property rights • Produces the core Python distribution: the Python language, standard libraries, documentation, source code, etc. • Manages contributions to the Python codebase

35. The PSF will outlive Guido. There is no guarantee of any commercial vendor's long-term survival. Oh, by the way... you do have software escrows for all of your commercial software don't you?

36. Is OSS as good as commercial software? • Vendors and products vary widely in both the commercial and open-source arena. • The fact that a piece of software is commercial is no guarantee of its quality. • The best open-source software is as good as the best commercial software.

37. The Bottom Line • Each product and vendor should be evaluated on its own merits, regardless of whether it is commercial or open-source. • Some open-source software is in the same league as the best software anywhere, commercial or not.

38. What about market share?Open-source: mindshareHow alive is the user community? • International Python Conference • IPC - in USA since 1994 • In conjunction with OSCON since 2002 • EuroPython conference • in Europe since 2002

39. NewsgroupActivitycomp.lang.* December 2002 java 26953 c++ 19913 c 13874 perl 10486 python 9647 basic 7909 ruby 6466 lisp 6132 tcl 5256 pascal 4229 smalltalk 2398 fortran 2355 cobol 1845 Statistics compiled by Aaron K. Johnson.

40. Python is famous for its "batteries included" philosophy What are its capabilities, features? Frank Stajano

41. GUI strings regular expressions database connectivity HTTP, CGI, HTML, XML numeric processing debugger object persistence The Python Standard Library

42. What about execution speed ? Slower than a systems language, but ... • Modern hardware processors generally make language speed a non-issue • Most applications are limited by speed of database or network connection, not programming language Remember when they said that relational databases would never catch on? "Too slow" they said...

43. How will we ever survive without type-checking? I found that type issues simply never arose. My unit tests kept my code on the straight and narrow. I simply didn't need static type checking. Robert C. Martin

44. Where will I find support? • comp.lang.python -- Outstanding!! • Many online books and tutorials • Consulting and contracting firms that specialize in open-source training and development

46. The Bottom Line... "Use the Best Tool for the Job: Put Both a Scripting and Systems Language in Your Toolbox" - Bill Venners http://www.artima.com/commentary/langtool.html

47. How Do I Start? • Investigate scripting and agile languages • Navigate your standards process, if necessary • Provide training, learning materials • Start experimenting .... • Scripts, internal/administration utilities • One-time, throw-away programs • Test scaffolding • Prototyping

48. A prediction: you will move “The Tipping Point” • It is possible to write programs to automate every task. But you don’t. • Python makes it easy enough