Getting More Project Documentation from Free Software Communities

1. Getting More Project Documentation from Free Software Communities Andy Oram — andyo@oreilly.com http://www.praxagora.com/andyo/

2. Some of my editing work

3. Overview • Community documentation—what it is, and what are its strengths and weaknesses • Professional publishing—its strengths and weaknesses • How to improve community documentation—communities and publishers together

4. Overview • Community documentation—what it is, and what are its strengths and weaknesses • Professional publishing—its strengths and weaknesses • How to improve community documentation—communities and publishers together

5. Community Documentation • Produced by users for each other • Could be as small as a posting to a mailing list; could also be blogs, web pages, wikis, HOWTOs, full sets of manuals • May be supported by advertising, print publication and sales, subsidies, or just a "thank-you!" • The conversation is just as important as the information; knowledge is built collectively through interaction and feedback

6. General problems with documentation • Confusing readers with abstractions • Drowning readers in details • Not understanding what the reader needs • Scrambled organization

7. Confusing readers with abstractions Customizations can take several forms. Launchers facilitate the startup of stand-alone desktop applications.

8. Confusing readers with abstractions — improved The desktop offers several ways to save time and make work easier. If you find yourself using an application regularly, you can put a button on your desktop that starts the application with a single click; this is called a launcher.

9. Drowning readers in details In the Command field, enter the pathname that would be used to start the application from the command-line, which can be either absolute or relative to the path set in your shell and can be followed by arguments.

10. Drowning readers in details — improved You can create an icon on your desktop that lets you start, with a single click, any application that can run on your system. Basically, you need to tell the icon to run the same command you would use if you were starting the application from the command line. Even if the application is a script, or requires arguments on its command line, or has no graphical interface of its own, you can start it from an icon this way...

11. Not understanding what the reader needs The Command field is required so that the launcher can find the application. This command is simply the filename of the application's executable file. It can be specified as any valid pathname.

12. Not understanding what the reader needs — improved Every application is a file; the name of that file is what you need to put in the launcher's Command field. You can probably find that name in the documentation for the application. If the application is available through a launcher or menu item, you can check its Properties...

13. Scrambled organization Select the action you want a key to invoke, then press the combination of keys that will invoke the action. Be forewarned that if you choose a shortcut that has already been defined by an application, you can no longer use that shortcut in the application because pressing the shortcut will invoke the action from this box instead. A key can be pressed in combination with the Shift, Control, and Alt keys...

14. Scrambled organization — improved You have a wide variety of key combinations to choose from, because you can use any key that produces its own action (a letter, a number, punctuation, or a function key) and can press the key by itself or with any combination of the Shift, Control, and Alt keys. Be forewarned, though, that if you choose a key combination that has already been defined by an application...

15. General problems on a larger scale • Confusing readers with abstractions • Drowning readers in details • Not understanding what the reader needs • Scrambled organization

16. Confusing readers with abstractions • It's hard to know that a particular document solves your problem • A writer often assumes that readers share the same goals as the writer, and therefore doesn't articulate the goals for reading and using the documentation

17. Drowning readers in details • Much project documentation simply lists steps to take, lacking the background needed to understand what's happening • Many postings and short documents, which are written for a particular problem at a particular time, leave out context

18. Not understanding what the reader needs • Developers are (rightly) proud of what they've accomplished, so they boast about their accomplishments instead of telling why and how to use the tools • Many subjects are covered over and over in multiple documents, while other important subjects are undocumented

19. Scrambled Organization • Searches are difficult and frustrating—but people asking questions are ridiculed for not finding information • How do you know whether to trust the information you do find? • Readers often don't know what background they are expected to possess before reading a document • There is no clear path from one document to another • Timeliness has good and bad points—a posting may be perfect for certain readers, but quickly becomes obsolete (yet the posting stays in an archive forever and turns up on searches)

20. Overview • Community documentation—what it is, and what are its strengths and weaknesses • Professional publishing—its strengths and weaknesses • How to improve community documentation—communities and publishers together

21. What are publishers good for? • Paying authors—this motivates them to work harder and may even give them the time they need to do a good job • Conveying prestige—who wouldn't like his name on the cover of an O'Reilly book? • Marketing and distribution—promotes the technology as well as the book • Ensuring quality

22. Ensuring quality • Publishers are still the experts in ensuring quality in the following ways: • Selection of material and authors • Editing • Artwork, design, indexes • Result: quality is still the prime value added by publishers—when publishers do their job

23. Challenges to traditional publishing • Many topics seem to be moving from books to the Web (particularly reference material and advanced topics) • Profits for online docs are less certain, so persuading authors to write them and publishers to publish them is more difficult • The Internet provides many new outlets for expression: a thoughtful weblog, updated frequently, brings prestige • Readers increasingly rely on searching and reading reviews, instead of traditional marketing

24. So—are publishers still relevant? • Given the problems of community documentation, professional involvement is still crucial • Quality—particularly in editing—is where publishers add the most value • A partnership between publishers and the community can provide the way forward • To support professionalism, community documentation needs funding

25. Publishers online • Early experiments with online distribution before publication, and free documentation • Subscription services (Safari) • Rapid development

26. Online distribution and free documentation • Manpages, Free Software Foundation manuals, etc. created formats suitable for both online and print publication • 1994: John Ousterhout's Tcl and the Tk Toolkit goes online before publication, due to popular demand • 1995: Olaf Kirch's free LDP project, Linux Network Administrator's Guide, is edited and produced by O'Reilly

27. Subscription services • http://www.safaribooksonline.com • Offers whole books translated into HTML format • Not a new paradigm, but valuable (and well-liked) as a new delivery mechanism for existing content • O'Reilly launched Safari in July 2001; we were joined by many other publishers and became profitable two years later • SafariU: build your own low-cost textbook • Conversion is time-consuming and inhibits updates

28. Rapid development • Books are developed using techniques comparable to those used for sophisticated software development • Posted online as they're being written • Incorporate reader feedback (i.e., bug reports) immediately • Used by software projects, such as MySQL • Pioneered for books by The Pragmatic Programmers • O'Reilly RoughCuts

29. The Pragmatic Programmers • Programmers first and foremost; book publishing is just one expression of the site's information sharing and community • Allow authors to update books up to the day they go to the printer: increases technical accuracy and timeliness, while sacrificing some aesthetic appeal • Reader feedback improves books and evolves into ongoing updates to web sites • Conventional editing as well

30. O'Reilly RoughCuts • http:// www.oreilly.com/roughcuts • Books are written online and accept user comments before publication, as with Pragmatic Programmer • Books suitable for this treatment: • Able to be written quickly, over a matter of months • Topics are fast-changing and of urgent interest • Traditional editing is still central

31. Overview • Community documentation—what it is, and what are its strengths and weaknesses • Professional publishing—its strengths and weaknesses • How to improve community documentation—communities and publishers together

32. How to improve community documentation • Pay authors and their helpers • Extract and formalize the best of the informal help • Rate documents • Organize documents

33. Pay authors and their helpers • Set up a consortium to collect and distribute payments • Consortium could be funded by computer vendors or user dues • Board of directors or a rating system could reward good documents

34. Extract and formalize the best of the informal help • Readers on mailing lists sometimes create documents based on list discussions • It would be valuable for editors to work on the best and most popular of these documents • Some published books grow out of authors' experience answering questions on mailing lists. O'Reilly examples: • Running Linux • Mastering Regular Expressions

35. Problems of making payments by boards of directors • Everyone brings prejudices and assumptions to their reading • Who has time to read all the documents? • Directors probably have different backgrounds and habits from the audiences for whom documents are written • A person cannot tell whether a document is useful merely by reading it

36. Rate documents • Proposal for a formal, official system • Proposal for an instant feedback system

37. Proposal for a formal, official system • Require user registration • Proof of work to screen out bogus and repeat registrations • Perhaps type in parameters when installing software • Register when purchasing a computer • Set up a rating community • Vote on documents • Indicate background of user while rating • Who wants to invest that much effort?

38. Proposal for an instant feedback system • Provide a quiz at the end of each document (probably not chosen by author—independent editors or panel should choose) • Incentive to take quiz: reader would want to know whether he or she understood the material • Cheating isn't worth the effort—might as well read the document • Documents whose readers scored well on the quiz would be considered good documents • Funds could be distributed to writers and organizations that created good, popular documents

39. Organize documents • Different documents are good for different readers—there are educational and cultural differences • User could register self • Example: "I am new to databases" • Example: "I have administered MySQL but am new to PostgreSQL" • Author or web site registers document • Example: "This document is for database newbies" • Example: "This document is for MySQL administrators new to PostgreSQL" • Web sites help users navigate through collections of documents

40. Conclusions (take-aways) • Community documentation is crucial, because ever-changing software requires immediate explanations from people who are actually using the software • Communities also need professional help, to create longer-lasting documentation that can stand on its own • Vendors and users should fund consortiums that rate documents and reward those who work on them Andy Oram — andyo@oreilly.com http://www.praxagora.com/andyo/