1 / 14

Database Publication Practices

This publication discusses the increasing submission rate in conferences and the challenges faced by researchers. It provides observations, perceptions, and suggestions for coping with the growth in conference submissions. The suggestions include rethinking the role of conferences, reducing the number of papers and increasing acceptance rates, and making tenure evaluation based on paper quality. The publication also addresses second-order issues such as double-blind reviewing and accommodating growth.

theresac
Download Presentation

Database Publication Practices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Database Publication Practices Surajit Chaudhuri Microsoft Research

  2. Coping with Growth • Issue: Higher submission rate • We are not alone in facing this problem • What are others thinking? • ACM SGB Task Force on the Impact ofIncreasing Conference Submissions (Chair: Alexander L. Wolf) – next 4 slides • My analysis and some painful suggestions

  3. ASE (1995-2004) DAC (1998-2004) DATE (2001-2005) EComm (1998-2005) FOCS (1990-2004) FSE (1993-2004) GLSVLSI (2002-2005) ICSE (1992-2005) ICSM (1999-2004) ISSTA (1989-2004) Middleware (2000-2004) MobiCom (1995-2004) MobiHoc (2000-2004) MobiSys (2003-2004) PODC (1994-2004) PODS (1993-2003) POPL (1973-2005) SCG (1995-2004) SIGCOMM (1998-2004) SODA (1990-2005) SPAA (1995-2003) STOC (1991-2005) UML (1999-2004) UAI (1992-2004) Basis of SGB Task Force Data

  4. MobiCom (1995-2004)

  5. Observations/Perceptions: Stature/Workload • Conferences with lowering acceptance rates receiving more polished papers on narrower topics • Pressure to publish in top venues • CS argued that conferences more important than journals; now we are suffering for it • grad students expected to publish in top places • Between a rock and hard place • grow PC, but lose coordination • shrink PC, but lose quality of reviews • Vicious cycle • people agreeing to do more PCs (can’t say no) • people do less per PC

  6. Most Radical Idea • Rethink the role of conferences • reduce importance w.r.t. journals • reduce number and increase acceptance rates • conference presentations derived from “best” journal submissions (rather than vice versa) • make tenure evaluation based on quality of top five papers, not number of papers • Journals have better scale properties • larger reviewer pool • less time pressure on authors and reviewers

  7. My Analysis and Suggestions (No implication for SIGMOD06)

  8. Request to the Steering Committees • Experimenting with procedural changes • Observe • Make only one significant change for a while • Observe • Avoid second-order changes that do not address the pain point

  9. Second Order Issues • Double-blind reviewing • Expected to impact on selection of “border-line” papers • Many ACM SIGs follow • SIGGRAPH, AAAI, SIGCOMM • Author Feedback • Good for “venting” • Tight timeline for reviewing makes it ineffective

  10. Accommodating Growth • Acceptance rate • Acceptance rate is 15%, 20% or 39%? • Do the math! • Not lower than many other large conferences • Mandated changes seem unreasonable • Diversity and Narrowing of Topics • AAAI: “Big Ideas”, Tech papers, Abstracts • Independent conferences as tracks (like WWW) • Reuse journals as publication • Program Constraints • Presentation decoupled from acceptance • Posters and Plenary (old KDD style)

  11. Impact on Reviewing Infrastructure • Choice of PC members crucial • Quality of Papers highly correlated with PC members • “Everyone is a PC member” • VLDB05: 610 reviewers • Large PC has likely to have high variability • VLDB05: 610 reviewers • But, otherwise 3-review load is too high • Suggestions (next 2 slides) • Short-cuts in reviewing process • Throttle the “flow” of papers

  12. Reviewing Quality/Load • 2-level PC (but reduce dependence on external reviewers) [like AAAI, SIGGRAPH] • Senior PC member nominates • Handles 4/5 papers • Good training for future PC experience • Early Rejection • If a paper gets 2 “weak” rejects, it is rejected without a third review • General Chairs should yield more time to reviewing • Electronic PC requires it • Electronic Proceedings make it easy • We should learn/coordinate with broader CS community

  13. Throttle the flow • No other ACM SIG engineers pipeline • SIGGRAPHICS or EuroGraphics • Mobicom or SIGCOMM • Takes motivation away to make serious changes • Selectively “break the pipeline” • VLDB-> ICDE • SIGMOD -> VLDB (except for roll-over papers) • Send the right (not polite) message • Weak Reject” vs. “Reject” • Very short reviews for bottom 10% - 20%

  14. More Serious Issues beyond Publication Practices • Legacy of mid’80s – mid 90’s • Parallel DBMS, Query Optimization • Debate between deductive DB and OODB washed away! • Jury is still out on the last decade? • Today: • Are there too many problem statements? • Everyone working on “personalized” problems • Many “fuzzy” problems with “fuzzy” yardstick for solutions? • Worry: Weakening link to systems/hard engineering

More Related