INTERACT 2011 PC Meeting Nicholas Graham and Nuno Nunes Technical Program CHairs

1. Nick Nuno INTERACT 2011 PC MeetingNicholas Graham and Nuno NunesTechnical Program CHairs March 10, 2011, Veszprém, Hungary

2. Tracks

3. INTERACT 2011 Tracks • January 10+24 submission • Full papers abstracts + papers • March 27 submission • Workshops, tutorials • April 7 submission • Short papers, industrial program, doctoral consortium, SIG discussions, posters • At conference • Student design competition

4. INTERACT 2011 Tracks • January 10+24 submission • Full papers abstracts + papers • March 27 submission • Workshops, tutorials • April 7 submission • Short papers, industrial program, doctoral consortium, SIG discussions, posters • At conference • Student design competition Why such an early date for full papers? Want to allow authors to resubmit rejected full papers as short papers.

5. Reviewing Process Full Papers Track

6. Reviewing Model • INTERACT 2009 and earlier • Program Chairs recruit reviewers for each paper • Small group makes final selection based on reviews • System unworkable as number of papers increases • Currently > 400 papers to review • No longer possible for PCs to read every paper • This year adopted Associate Chair model • Distributes reviewing and decision making to sub-chairs, each responsible for a selection of the papers • Increasingly widely used in HCI conferences

7. Associate Chair Model • 77 Associate Chairs • Each responsible for a group of papers (up to 7) • Recruits reviewers for those papers • Total 693 external reviewers recruited • Leads discussion among reviewers • Recruits extra reviewers if reviews very divergent • Writes meta-review summarizing reviews • Makes accept/reject recommendation to chairs

8. 528 Abstracts Received Jan 10: Abstracts Due Jan 10-Jan 19: Nick and Nuno assign papers to ACs 1 primary AC, 1 secondary AC 2 external reviews + 2AC review = 3 reviews for each paper Jan 19-Jan 24: ACs recruit external reviewers for papers Jan 24-Feb 21: 2ACs and external reviewers review papers

9. 1ACs initiate discussion with reviewers, recruit extra reviews if necessary Feb 21-March 7: Primary ACs prepare meta-reviews Jan 10: Abstracts Due Authors get to see reviews Jan 10-Jan 17: Nick and Nuno assign papers to ACs March 7-March 14: Authors prepare rebuttals March 14-March 21: Online discussion ACs can see all papers and reviews Jan 17-Jan 24: ACs recruit external reviewers for papers March 21-28: Nick and Nuno finalize decisions Jan 24-Feb 21: 2ACs and external reviewers review papers March 28: Notification letters mailed

10. Reviewing Model Other Tracks

11. Reviewing Model • Short Papers • Will also use AC model • Other tracks • Small number of submissions • Track chairs assign reviewers directly

12. Reviewing Tool

13. Precision Conference Systems (PCS) • Very powerful tool for managing larger conferences • Now used for most of the larger HCI conferences • UIST, CSCW, CHI, ITS, IHM, Mobile HCI, … • Direct support for AC model • Supports chairs, track chairs • Persistent reviewer database – allows reuse, growth of database from year to year

14. Publication Schedule

15. Publication Schedule • Proceedings to be published by Springer • USB key • Optional physical proceedings • Springer must receive electronic version of complete volume by June 27, 2011 • Therefore camera-ready copy due: • Long papers by April 18 • Short papers by June 13

16. Submission Information Full Papers

17. Full Paper Submissions • Received 528 abstracts and 402 Full papers • In 2009 received 520 abstracts and 359 Full papers • 1.5% increase in abstracts • 12.1% increase in papers

18. Short Paper Submissions • Based on 2009, expect circa 200 submissions

19. Top 11 Keywords

20. Bottom 10 Keywords

21. Observation • Surprising that some areas very poorly represented • E.g., visualization, adaptive interfaces, HCI education • Are these declining areas? Or thriving areas that are well-served by other venues?

22. Expected Acceptances Full Papers

23. Range of Acceptance Rates Based on today’s numbers; will change after rebuttals, discussion Note there are 51 papers between 3.0 and 3.25. Differences this small are well within margin of error of reviewing process. Selection among papers in this range will be very difficult.

24. Rough Track Breakdown Assuming ~115 Full Papers, ~60 Short Papers

25. Monday Sept 5, Tuesday Sept 6 • Workshops, Tutorials and Doctoral Consortium • No prediction yet on how many we will accept • In 2009: • Workshops • 2 two-day workshops • 12 one-day workshops • Tutorials • 3 full-day tutorials • 2 half-day tutorials

26. Wed Sept 7 – Friday Sept 9 Papers Morning 9:00 Keynote 10:30 Break 11:00 Slot 1 12:30 Lunch Afternoon 13:30 Slot 2 15:00 Break 15:30 Slot 3 17:00 • 5-7 parallel sessions • Total 55 sessions • Typical Formats • 3 Full + 2 short • 4 Full

27. Keynotes

28. Keynotes • 3 keynotes (Wed, Thursday, Friday AM) • Suggestions welcome • Perhaps invite on of the “honorary chairs”? • Don Norman, Larry Constantine • Or perhaps create panel for honorary chairs

29. Awards Accessibility Award Brian ShackelAward

30. Accessibility Award • Process • Developed in consultation with Gerhard Weber • ACs have a box in the review form where they can nominate papers for the award • Nominated papers will go to award selection committee

31. Accessibility Award • Status • Only one paper nominated so far • 21 papers submitted with primary keyword “Accessibility and Usability” • Scores range from 1.00 – 3.73 • May be other relevant papers with high scores that did not use this primary keyword • 35 used “Accessibility and Usability” as secondary • Does not appear that committee will be overwhelmed with choice

32. Brian Shackel Award • We are open to suggestions re process

33. Lessons Learned

34. Abstract Submissions • Assigned papers to reviewers based on abstracts, but nearly 25% of abstracts never led to a paper • Waste of ACs’ time seeking reviewers • Disappointment and confusion for reviewers who were asked to review, but then no paper arrived • Many, many requests to submit papers when abstract submission deadline missed • We turned these down • Recommendation: no longer do pre-submissions of abstracts

35. Associate Chair (AC) Model • Largely a success, but some issues • Hard to recruit ACs • Reviewer fatigue • Vast majority of ACs worked in a timely manner • A few nearly derailed the process • Horst Hoertner, Nahum Gershon did none of their AC duties, required last minute scrambling • Henry Duh, Gerrit van der Veer > 1 week late with meta-reviews, caused delay in rebuttal phase

36. Associate Chair (AC) Model • Many ACs reported trouble finding reviewers • Reports of reviewer fatigue, difficulty recruiting “big names” • INTERACT reviewer database now has 695 registered reviewers, 1078 volunteered review slots • Recruited reviewers via TC 13 mailing list, CHI-ANNOUNCE mailing list

37. Associate Chair (AC) Model • Recommendations • Need to aggressively recruit for reviewer database • Continue to use PCS so reviewer database not lost • Don’t invite ACs with history of dropping the ball • Plan for a few ACs who may be very late

38. Keywords • Should revisit keywords • Keywords important to reviewer selection • Most popular need to be finer grained • Consider removing some of the least popular

39. Thoughts on Publications • Springer’s long lead time adds 2.5 months to schedule • When combined with desire to allow full paper rejects to resubmit as short paper, leads to very early submission date for full papers • Abstracts Jan 10 for Sept 5 conference • Producing publications is becoming an enormous undertaking for publications chairs • Especially when any overruns in schedule cut into their time • Maybe time to consider professional proceedings production?

40. Thanks to TC13 …for entrusting INTERACT 2011 to us …to many of you for your hard work as ACs and reviewers … for your help with this process