WG TOOLS Discussion

1. WG TOOLS Discussion March 17, 2002 IETF 53 Minneapolis, MN

2. Agenda • What is the problem? • Brainstorming • What are the requirements? • For WG chairs? • For Editors? • For WG members? • For the IESG? • Thinking about workflow: an example • Demonstrations • Draft management CGI: Dean Willis • Bugzilla: Robert Sparks • RT • IEEE Comment Database: Bernard Aboba • Inter-Group Tool (alpha) • W3C SSL tool • Next steps • Internet draft on WG tools? • IMC tools server

3. What is the Problem? • For WG chairs • Lots of open issues to concretely track (600+) • Capture resolutions of issues as well as the conversation (email thread) • Tracking lots of drafts (WG vs. individual submissions) • Keep up with deliverable dates (milestones, WG last call) • Keeps track of drafts in last call, can lose a draft or comments on it • Keeping track of design team work (who is on it, what they’re doing) • To track what topics are in each document; topic coverage • Specialists in different knowledge domains • Tracking stuff that fell off RFC Editor, IESG lists mysteriously • Keeping track of draft standard option implementation • Formalize WG last call process (not only nos but yeses) • Record issues to refer back and say it was resolved; integration with email and archives • Signoff process (not just people who are posting or speaking) • To track design group efforts • Automated capturing of issues that come up frequently into a FAQ

4. What is the Problem (cont’d) • For Editors • Conversion between differing styles • Collaboration between standards bodies • Issue X comes up again, want to go back to the discussion • Track changes to sections of draft since a date (not diff) • For WG members • Keeping track of draft standard option implementation • Link a resolution to the document section that it settled • Searchable email lists without having to download the whole thing • Track changes to sections of draft since a date (not diff) • For IESG • Want full data for IESG or WG chairs, summarized data for WG • Some comments that WG chairs can see, some that only IESG can see • IESG tools that work together with WG tools to track updates • Work item tracking integrated w/IESG tool

5. What Are the Requirements? • For WG chairs • Ability to determine who gets to input • Want to be able to delegate to authors • May want to restrict access to output, too • Calendar generation • Automated reporting for large efforts • For Editors • Versioning? • Collaborative editing tools? (XML to RFC?) • For WG members • Email interface as well as Web • Link to the WG archive • For IESG • Integration between IESG and WG tools • Long term archiving of the results (in standard format? Text?) • Design groups won’t generate email on mailing list so tool may have unique discussions

6. One WG’s Problem • AAA WG history • Mailing list discussion was diffuse • Lots of discussion, but… • Hard to tell if commentors were asking for a change to the documents or just asking questions about them • When changes were requested, hard to track status • Were changes made or was the request rejected? • Was the original requestor satisfied with the changes? Or was the issue still open? • Was WG consensus reflected in the documents? • Document status was hard to gauge • 3G folks ask: “When will you be done?” • Question was difficult to answer! • How many issues were outstanding? • How fast were issues being resolved? • How fast were issues being raised?

7. A “Solution” • A formal Document Change Request (DCR) process • To get an issue considered, WG member MUST • Post a message to the AAA WG mailing list with [Issue] in the Subject • Message body includes a description of the issue, the affected document (s) and the proposed change • Document number assigned to the issue • WG discusses the issue • Document editor proposes a resolution • WG chair verifies WG consensus • Issue status tracked by document on a Web page • http://www.drizzle.com/~aboba/AAA/issues.html

8. DCR Schema • DCR submission • Description of issue • Submitter name: Your_Name_Here • Submitter email address: Your_Email_Address_Here • Date first submitted: Insert_Date_Here • Reference: URL to e-mail describing problem, if available • Document: Document Requiring change [base, nasreq, mobileip, cms] • Comment type: ['T'echnical | 'E'ditorial] • Priority: ['S' Must fix | '1' Should fix | '2' May fix ] • Section: Insert_Section_Number_Here • Rationale/Explanation of issue: • Description of problem • Requested change: • Proposal • Issue status • Text Proposed - Text has been proposed on the list, and no consensus has been reached • Accepted - Text has been proposed, consensus has been reached, but the fix hasn't been included in a specification yet. • Pending - Discussion is on-going, no proposed text has been proposed • No Discussion - No discussion has been initiated on this issue. • Clarifying Text Required - The issue can be closed if additional clarifying text is added to the draft • Possible Reject - Issue will be rejected unless objections occur.

9. Better, but… • Issues now tracked formally • Now possible to distinguish between change requests and ordinary discussion • Editors, Chairs can focus on email with [issue] in the subject • But… • Lots of manual labor required • Issue #s manually assigned • Issue descriptions cut and pasted into Issues list • Issues web page manually maintained • Not as integrated as it could be • Issues web page not linked to WG discussion • Can’t click on an issue and see the discussion that ensued • Not easy to verify that WG consensus was reflected in the issue resolution • Errors possible • Issues sometimes fall through the cracks

10. An “Ideal” Solution? • WG member sends mail with [Issue] in the subject • An automated attendant assigns an issue number, replies to the WG mailing list • Attendant parses the body of the message, assigns the issue to a queue, depending on the draft • If the issue relates to multiple drafts, it goes in multiple queues • Editor receives mail that an issue has been assigned, with a URL pointing to a web page with all the issues assigned to him/her • Editor can get status on all issues on the web page • Issue list for each draft • Open versus Resolved issues • Details of each issue • Status of each request can be reviewed • Accept, Reject, Pending • If Rejected, reason for the rejection • Editors can resolve/assign issues • Can assign the issue to someone to come up with a solution • Can resolve issues • Original poster verifies the resolution • Editor can resolve the issue, but poster or Chair needs to “close” it • Reports can be generated • Incoming issues • Resolved issues • Open issues • Rejected issues • The whole process is secure • An attacker can’t delete or modify issues • Editors can’t touch issues outside their drafts

11. Draft CGI Demonstration Dean Willis

12. Bugzilla Demonstration Robert Sparks

13. RT Demonstration

14. IEEE Tools • IEEE has developed their own tools over several years • Solution derives from their definition of the problem, which is not the same as the IETF problem • IEEE process • Largely a batch process: comments are received in response to Ballot solicitations • Task Groups meet in person to review and resolve comments • Comments are reflected in the document, which goes to ballot again • IETF process is much more interactive than this • Still, productivity is impressive • Solution • Comment process is automated • Members MUST submit comments for each ballot or they lose their votes • Comments MUST be submitted in a rigidly defined format • Ballots typically submitted in a spreadsheet • To be valid, comment must specify the document and section, exactly what text is to be changed, and what it is to be replaced with • Vague comments like “make it better” are classified as invalid • Comments are automatically collated into a big spreadsheet or a database • WG comes to consensus on how to deal with the changes • Tools make it possible to resolve hundreds of comments within a few days • Can track action items assigned to WG members, send prod-o-grams

15. Feedback?