1 / 12

Two Issues Concerning Research Conferences

This article discusses two issues concerning research conferences: the lack of acceptance of Big Idea/New Directions papers and the potential threat of large-scale conferences on conference quality. It proposes solutions such as dedicating a session to Big Idea papers and creating an ad hoc committee to evaluate conference size and explore best practices.

mechelleh
Download Presentation

Two Issues Concerning Research Conferences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Two Issues Concerning Research Conferences Dave Patterson October 2004

  2. 1) Big Idea/New Directions Papers • Problem Trying to Solve: Given avalanche of submissions to conferences, Big Idea / New Directions papers rarely accepted at many conferences • Program Committee (PC) must reject 100-170 papers, easy to find bullet holes in such papers, they get a few low scores, and PC rejects them • Occasionally “hero” PC member gets 1 thru, but not sustainable (not just call for papers, PC chair) • Numbers making many conferences conservative • Given important stature of conferences today, Big Ideas / New Directions papers even more important now

  3. 1) Big Idea/New Directions Papers • Proposal:Try a 3 – 4 year experiment to attract such papers at some conferences • Mechanism:Separate Big Idea / New Directions PC • Former PC chairs + people with reputation for Big Idea / New Directions select ~ 3 papers • Dedicate 1 session to such papers • Evaluate value of Big Idea papers to decide if discard/keep/expand this approach: attendance at session, good discussions at conference, lead to subsequent papers, …

  4. 1) Big Idea/New Directions Papers • Our conference already accepts them • Great! Please share your experience, but no need to try this idea. • But we lose slots for 3 “good” papers? • For 3 years, let’s drop 1 panel session or 1 keynote (often nil in my experience) • Discuss! • Suggestions for conferences and/or people to contact as likely match?

  5. 2) Is large scale a threat to conference quality? • Background – Number of paper submissions increasing over time while number of papers presented roughly constant

  6. 2) Is large scale a threat to conference quality? Papers Submitted

  7. 2) Is large scale a threat to conference quality? Acceptance Rate

  8. 2) Is large scale a threat to conference quality? • Impact on Program Committee – Either more papers per member or larger PC • Either affects quality of reviews and of decision? • Impact on Authors of Rejected Papers – Inaccurate reviews  decision feels arbitrary, unfair • Resubmit them to related conferences or submit more papers per conference  more papers per conference • Impact on Conference,Field – Losing good ideas? • Are papers ranked 31-60 really that bad? (Its research) • Researchers don’t try riskier ideas since less chance?

  9. 2) Is large scale a threat to conference quality? • Proposal – Create an ad hoc committee to write report (& talk in Feb) on whether / when a problem including: • Why are numbers increasing? (More people in field, resubmiting, multiple papers/conf) • What size is a problem? (150? 250? 350?) • What have some conferences tried, and did it work or not? (best/worst practices) • New good ideas to help to cope with issue

  10. 2) Is large scale a threat to conference quality? • ExamplesofBest/Worst Practice • STOC – Went to dual tracks • PODC – 5 minute Brief Announcements (1 page in proceedings) + 25-min papers • 75 talks in 3 days, 50% - 50%, single track • SIGMOD – lower bound to accept rate, so more papers submittedmore presented • SIGGRAPH – add 5-minute recent result papers with later deadline, quick review • DAC? – Hierarchical Program Committees (divide by topic, allocate by track)

  11. 2) Is large scale a threat to conference quality? • Examples of New Ideas • No downside to submitting, so create downside by publishing authors’ batting averages to  • Limit number of papers one can submit to a single conference (as often a limit for PC members) • Since now many related conferences, if some submit papers to each, go to a common paper deadline to force authors to decide target • Journals cope with surge by having editors reject papers outright without further review, for otherwise surge would break review system: give PC chair same authority  smaller PC

  12. 2) Is large scale a threat to conference quality? • Request – If you think it’s a good idea, suggest names of people who might be willing to volunteer for ad hoc committee, possibly a chair? • Important to get people from diverse communities to collect practices, ideas

More Related