1 / 23

IEFT Last Call and Gen-ART IETF 72 WG chairs July 30, 2008

Mary Barnes (Gen-ART secretary). IEFT Last Call and Gen-ART IETF 72 WG chairs July 30, 2008. Overview. Background of Gen-ART Summary IETF-LC Process Gen-ART Role during IETF-LC: Reviewer assignment process Reviewer responsibilities Handling review comments Other Considerations Summary.

fauve
Download Presentation

IEFT Last Call and Gen-ART IETF 72 WG chairs July 30, 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mary Barnes (Gen-ART secretary) IEFT Last Call and Gen-ARTIETF 72WG chairs July 30, 2008

  2. Overview • Background of Gen-ART • Summary IETF-LC Process • Gen-ART Role during IETF-LC: • Reviewer assignment process • Reviewer responsibilities • Handling review comments • Other Considerations • Summary

  3. Gen-ART Background (1) • Original Gen-ART review team formed by Harald Alvestrand in 2003 (original secretary Avri Doria). • Assignments were made at time document was placed on telechat agenda • Objective was to aid the General Area Director to more effectively determine the readiness of documents • Original team was comprised of less than 10 active reviewers • Around 1380 documentreviews completed since 2004 - averaging 350/year

  4. Gen-ART Background (2) • Reviews were often sent right up to the start of the telechat: • Completing reviews was done under a “best effort” model • Many reviews were sent only to gen-art mailing list • Review guidelines evolved based on feedback from reviewers • Oftentimes, recipients of reviews were not certain how they should be handled • In general – a positive response to reviews • IETF Last Calls added as a “nicety” or “wbn” in 2005.

  5. Gen-ART Evolution (1) • Kicking off the review process for all documents during IETF LC was initiated by Brian Carpenter • Overall, this has improved quality in terms of the number of documents that are deemed ready at Telechat time, based on 2007 data: • 250 documents assigned at IETF LC time • 80% of the last call reviews completed (205 documents) • Around 25% of the docs are deemed “Ready” at LC time. • 75% of the docs were ready at Telechat time (144 documents) • For the docs not reviewed at LC time, around 20% were ready at the time of the telechat.

  6. Gen-ART Evolution (2) • Gen-ART and other directorate/review team reviews (e.g., Secdir) are now expected - the reviews are received, considered and responded to along with other LC comments. • Many ADs now consider the reviews as input in forming their decisions.

  7. IETF LC Process Overview • Gen-ART does not add steps to the process. • Input is part of the IETF Community Review in the form of “Comments, Suggestions” in a std manner. • Gen-ART reviewer is very often involved in resolution of “Concerns”, but reviews are non-blocking although Gen-ART ADs consider reviews in forming their positions.

  8. Gen-ART: Reviewers • Current Team is comprised of around 13 individuals: • Past and present WG chairs • Past and present IAB members • Key technical experts from various WGs/areas • Past ADs/IETF chairs • 5 Reviewers only receive one document assignment per month • List of current/past reviewers: http://www.alvestrand.no/ietf/gen/art/reviewer-list.html List is dynamic as reviewers’ day job workloads vary. • Current size is due more to availability of reviewers than any objective to limit size of team. • Estimate that 32 reviewers is optimal.

  9. Gen-ART: Reviewer Assignment • During the week IETF LC is announced a Gen-ART reviewer is assigned by the secretary. • Assignments are made on Thursday evenings CST. • No attempt is made to assign reviewers based on expertise: • This has been requested in the past and nacked. • Only reasons why the next reviewer (per round robin process) doesn’t get next doc is due to: • WG chair • Individual involvement as contributor, WG reviewer, etc. • Reviewer doesn’t have the bandwidth

  10. Gen-ART: Reviewers’ reponsibilities • Reviews are due on or before the IETF LC deadline. • Reviews are always sent to Gen-ART mailing list and typically to author, WG chair and responsible AD. • May additionally be sent to IETF list. • Reviews are not typically sent to WG mailing lists. • However, it may be very, very useful for PROTOs/Authors to do so in some cases to ensure WG support for resolution of concerns. • Reviewer is responsible for re-reviews of any document updates, in particular when the doc comes up on the IESG telechat agenda.

  11. Gen-ART Review Details • Reviews are preceded by the standard template which provides the recipients a link to the Gen-ART FAQ along with a simple statement as to the handling of the comments. • Review template contains a summary, which is some variation of the following: • This draft is ready for publication as a [type] RFC, where [type] is Informational, Experimental, etc. (In some cases, the review might recommend publication as a different [type] than requested by the author.) • This draft is basically ready for publication, but has nits that should be fixed before publication. • This draft is on the right track but has open issues, described in the review. • This draft has serious issues, described in the review, and needs to be rethought. • This draft has very fundamental issues, described in the review, and further work is not recommended. • Unfortunately, I don't have the expertise to review this draft.

  12. Gen-ART Review Details • All substantive comments must be included in the public review. • Editorial nits may be identified along with other issues or they may be sent privately. • The review should apply generally agreed IETF criteria, such as: • [RFC1958] The Architectural Principles of the Internet • [RFC3426] General Architectural and Policy Considerations • [RFC3439] Some Internet Architectural Guidelines and Philosophy • [NITS] The "I-D Nits" document maintained by the IESG • [rfc2223bis] Instructions to RFC Authors • [BCP26, RFC5226] Guidelines for Writing an IANA Considerations Section in RFCs • [RFC3552] Guidelines for Writing RFC Text on Security Considerations

  13. Handling of Gen-ART IETF-LC reviews • Handling of comments is always at the discretion of the AD/WG chairs/PROTO Shepherds. • In general, unless a document is deemed “Ready” (or “Ready with nits”), a response to the comments (typically by the author/editor) is sent to the original email recipients within a reasonable time period. • These email exchanges can be very important for resolving issues. • It’s often very helpful for author/reviewer to come to agreement on proposed changes for some of the more important concerns raised in the review.

  14. Updating document in response to Gen-ART reviews • It is not recommended that a document be updated unless the revision is agreed with ADs/WG chairs/PROTO shepherd. • Document authors/editors must also consider other review teams that provide comments at IETF-LC time such as Sec-dir. • Very useful if a Gen-ART reviewer is able to review an intermediate version prior to the document being added to the telechat, however, this is entirely optional. • While reviews are non-binding, General Area Director does consider whether valid concerns have been adequately addressed.

  15. Gen-ART reviews for other docs • Not all documents go through IETF-LC since it is optional for Informational documents and individual/AD sponsored documents. • Reviews at the time of telechat are done in a similar manner in terms of process. • However, some differences: • Shorter review period for reviewer. • Much less time for interaction between reviewer and AD/WG chairs/PROTO/author. • Reviewer involvement is more variable depending upon the AD’s consideration of the review and nature of review summary

  16. Additional Gen-ART Info • Gen-ART process document: • http://ietf.org//draft-doria-gen-experience-02.txt • Gen-ART repository: • http://www.alvestrand.no/ietf/gen/reviews/ Note: plans to move this to IETF hosted server very soon. • Some files on a temp-dir at: www.softarmor.com/rai/temp-gen-art • Gen-ART mailing list: • NOT recommended that folks subscribe to this list. • List administrators (Mary Barnes and Brian Carpenter) add folks to the whitelist at time of first posting: • As of July 2008, around 313 folks on the whitelist (excluding ADs and Gen-ART members)

  17. Moving Gen-ART Forward • Automating tools is a priority for the near term. • May leverage reviewer assignment tool used by secdir • Process may apply to other areas: • Modified forms of this process (in terms of “tools”) have been applied to other area review teams and WGs • Open Office spreadsheet is very handy: • Easy to sort information and gather statistics • Facilitates review information for General Area Director • Facilitates secretarial process in assigning, caching and summarizing reviews • Aids reviewers in determining outstanding reviews. • In general, reviewer assignment would likely be the primary common tool. • Other applications tend to require more manual intervention, however, the frequency of reviews is much lower.

  18. Summary • While Gen-ART increases the quantity of review comments for documents during IETF Last Call, the overall improvement in quality makes the process worthwhile. • Document editors and WG chairs should always expect a Gen-ART review for their documents: • Info always available in the assignment spreadsheets. • Ping secretary if you don’t get one. • In the long run, tools should greatly facilitate the process and dissemination of information • Review team is very lean right now: • Could always use new reviewers. • WG chairs might consider and encourage conscientious WG reviewers/participants/leaders for the role. • Additional reviewers add minimal overhead to secretary’s job

  19. Questions/Comments • Any general feedback or suggestions for improvements?

  20. Additional Info on IETF LC Process • Good Overview/Reference from IETF-70: http://www.ietf.org/proceedings/07dec/slides/doclife-0.pdf • A few relevant slides included in the subsequent charts: • Shepherd’s Role: • < 50% of the docs seem to have PROTO shepherds versus AD’s . • PROTOs aren’t documented as part of IETF-LC process • IETF Process Overview • Handling of Review comments at LC time

  21. Additional Info on IETF LC Process

  22. Additional Info on IETF LC Process

  23. Additional Info on IETF LC Process

More Related