CMS Software Documentation Systems

1. CMS Software DocumentationSystems Ianna Osborne Lucas Taylor Lassi A. Tuura Johannes-Peter Wellisch Stephan Wynhoff For the CMS Collaboration Lassi A. Tuura, Northeastern University

2. Overview • Web Pages • Code Cross-Reference • User Guides • Tutorials • Architectural Documents • Project Planning Documents • Processing Systems • Interactive Web Servers • To Come • Thoughts

3. Web Pages • Project web pages / templates are part of the project’s documentation subsystem • All project web pages (not just documentation) versioned per release • Apply a common look to all web pages when building the documentation • Permanent repository of essential development knowledge/hints • Installation instructions, references to project download servers, etc. • Release notes and such come very easily • Links to rest of the documentation • Simple but very useful • A simple perl script that copies HTML files, makes simple substitutions and applies a common structure (itself a template that is read in) • Not yet used in all projects • Very successful in the projects in which it is used

4. Code Cross-Reference • Code documentation is embedded into the source code • Versioned documentation for each release • Process code to the produce a web: as up to date as it can get • Most aspects of source code covered: subsystems, directories, files, classes, methods, macros, … • Having considered a number of systems, chose doxygen • Documentation overhead is very small • High-quality output, very complete, easy to navigate • Capable, used widely, lively development • Allows documentation of independent projects to be merged(we are now working on generation/merging docs for our externals) • Successful, used and liked a lot • Need consistent encouragement for code authors to write meaningful comments and to maintain existing ones up to date

5. Example

6. User Guides • In the documentation subsystem of each project • Versioned documents for each release • Documentation subsystem provides • Document assembly logic • Prefix and initial sections • Trailer • Glossary, bibliography • Other packages provide “section inserts” • Merged into the shell at to provide the meat in the middle • Simple but functional convention • Produces both a web and a printed version (~150 pages)

7. Tutorials • Have organised a few tutorials on CMS software • ~A week long, ~30 people trained each time • A fair amount of preparation, but very successful • Getting essential feedback—not just on training, but software as well • Good for recruiting users/developers: lowers doubt barriers • The following format seems to have worked best so far • Gradually from simple usage—how to find the software, how to run commands—to more, even doing real analysis and visualisation • No long lectures: teach a subject, work on it, teach some more, … • Everybody in front of the computer all the time • Teachers are available all the time, including the exercises • Make sure everybody gets somewhere (= good/right results) • Planning 3-4 tutorials each year, changing subjects

8. Architecture Documents • The CMS CAFÉ project maintains • A database of requirements, use cases, scenarios, constraints, designs… • A collection of document templates for describing various views • A collection skeleton documents that describe a particular aspect, and pull in or link to the various “shell” fragments (requirement, …) • (See also J.-P. Wellisch’s paper!) • Gave up on LaTeX—too hard, too many processings required • Use XML instead: most content in simplified DocBook, own DTD to describe the shell fragments (whose actual text content is DocBook) • Agressively hyperlinked: the entire document set always has consistent structure, embedded fragments, cross-references and hyperlinks • Environment provides common functionality • Automatically generated glossary, bibliography (used subset of global) • Indexing, image conversions, … everything one generally needs

9. Architecture Documents… • CVS repository structure • Common stuff (templates, biblio, glossary, XML definitions, processing) • Shells: requirements, use cases, scenarios, constrains, … (directory/kind) • Documents (directory/document) • Very powerful documentation concept and a capable system • We can now actually guarantee that all references to requirement X-Y-Z have all the same number, the same text, and all point to the same, unique source where the user can find out more about the requirement • Can slice and dice descriptions when importing/linking to shells: can import only the parts that relevant to the importing context • Easy to build all sorts of summaries • But only beginning to make use of it for real documents • If successful, parts of the model may see use in the other projects • But see “Thoughts” slide as well!

10. Architecture Documents…

11. Project Planning Documents • Starting to build a project planning database not unlike CAFÉ • Task coordinators provide the necessary information • Used in scheduling and resource planning • Collecting and maintaining information about tasks • Task timing • Resources required • Task relationships • Currently producing simple reports • Full task and deliverable descriptions, resource summaries,… • Trivial CSV file for schedule: can import into MS Project, FastTrack,… • Evolving a project management DTD; no well accepted standard • An interesting prototype, may integrate or cross-link with the CAFÉ documents and perhaps with XProject (by Torre Wenaus)

12. Processing Systems • Code cross reference (source: C++) • Doxygen outputs web pages and class inheritance diagrams with graphviz (could also produce LaTeX and man pages, we don’t need those) • User guides (source: LaTeX) • LaTeX + dvips (PS) + ps2pdf (PDF); latex2html (web pages) • Whines: latex2html is… uh, very limited • Tutorials • Normal presentation tools (PowerPoint, StarOffice) • Example code + documentation released with the projects • Architectural documents (source: XML) • Norman Walsh’ DocBook XML/XSL stylesheets + local customisations, Saxon 6.x (XSL processor; to web pages and XSL-FO), PassiveTeX from TeXlive 6 (XSL-FO to PS/PDF with xmltex + passivetex + latex + dvips or dvipdfm); image formats PNG/EPS (gimp for conversions)

13. Interactive Web Servers • SCRAM has its own servers for release download and information on external requirements • LXR (Linux Cross-Reference) for browsing code • Very impressive tool for browsing and searching code • Complements Doxygen nicely • Updated automatically • Twice a day to re-index snaphots • Nightly to index new releases • Weekly to rebuild everything • Installed and operational for all projects; not yet publicly available • Other services under discussion • cvsweb and bonsai: browsing the CVS repository/history • Continuous snapshot build system (already have source and docs) • Problem tracking system

14. To Come • Web workbooks/guides • Recognise the need for more than just the current user guide • Lacking manpower • FAQs • Maybe use FAQ-O-Matic? • Moving the infrastructure into a separate project • Working towards “cookie-cutter” projects, documentation included • Generating web pages automatically for all projects • A cool acronym !

15. Thoughts • It is now fairly clear that XML/XSL is the future way to go… • XML processing systems are maturing • Outstanding HTML/Web output • Print quality improving—getting close or better than LaTeX • DocBook is in many ways more sophisticated than LaTeX • Shows as use of the tools in many large software projects • Linux & FreeBSD document projects, KDE, GNOME, … • Concerned about authoring cost in our community • Almost everybody is familiar with LaTeX • How to train writers to use Simplified DocBook? • XML is certainly reasonable possibility in projects that have fewer developers who are keen on trying things out—but will it work in large projects with dozens or 100+ developers? (Or when will it work?) • Assessing quality and authoring ease in smaller projects first • If successful, will discuss among developers in the larger projects