490 likes | 522 Views
A typical weak paper. Short and naïve introduction – demonstrates lack of background research and of expertise from the authors Methods / algorithms not very original – demonstrates lack of understanding of the state-of-the-art in the field
E N D
A typical weak paper • Short and naïve introduction – demonstrates lack of background research and of expertise from the authors • Methods / algorithms not very original – demonstrates lack of understanding of the state-of-the-art in the field • Results show operation of system on one example case – lack of systematic study demonstrates laziness and greatly reduces belief that research described is generally applicable • Short discussion limited to own research rather than putting work into perspective by comparing to previous studies – shows lack of knowledge of others’ work and reduces credibility CS 597
A typical weak paper: summary A weak paper is one where the authors describe work that is not very new, is not thoroughly validated, and is not properly placed in perspective with respect to previous work. Mostly, it is weak because… the authors have been lazy and have not done proper background research. CS 597
A typical strong paper • Comprehensive expert introduction – demonstrates extensive background research, mastery of the whole field, understanding of the important issues, and clear positioning of the research as new • Methods / algorithms are original – demonstrates good understanding of the state-of-the-art in the field, and expertise • Results include thorough quantitative validation – the authors seriously stand behind their research and make efforts to prove how it is new / different / better • Discussion puts work into perspective by comparing to previous studies – shows expertise and credibility by not being shy about comparing to other research CS 597
A typical strong paper: summary A strong paper is one where the authors demonstrate that they have complete expert understanding of the major open research issues in the field, and where they provide a convincing argument that they just professionally cleaned up one of those issues. Mostly, it is strong because the authors know what they are doing! And because they worked extremely hard on their research. CS 597
Introduction; Originality • Your paper needs to bring something new; • But research is nowadays a very active field, and chances are that something very similar to your contribution has already been proposed 20 years ago. • Again, don’t try to mask that fact, hoping that reviewers will be ignorant. On the contrary, use extensive literature research to frame your work comparatively to previous studies. • It’s ok to write: • “XXX et al. (1972) have proposed […], an idea which we here sought to further explore…” • “Our approach is related to that of YYY et al. (1995), …” • “We follow the hypothesis of ZZZ et al. (1938) and investigate …” CS 597
Discussion • Don’t try to hide weaknesses; readers and especially reviewers are not naïve and will explicitly ask you about those weaknesses if you don’t appropriately discuss them – in the end, you just waste time. • It’s ok to have in your discussion statements like: • “a clear limitation of our approach is that in its present from it cannot …” • “our study has not addressed the problem of…” • “better methods have been proposed for [component X] of our system …” • Just be honest, fair, objective. CS 597
Video compression for human viewers • Many optimizations are possible and some are already implemented by modern video codecs: • Texture complexity • Spatial and temporal masking • Importance of motion • Human contrast sensitivity function • … [e.g., Osberger et al., 1997/1998/1999] • But some remain which are more difficult to implement: • Importance of humans, faces, animals, … • Biological motion • Object size • Main actor vs. secondary actors • Man-made vs. natural objects • … CS 597
Purpose of the study To evaluate applicability of a biological model of visual attention to automatic prioritization of video Systematic study of 64 variants of the model Demonstration using MPEG-1 and MPEG-4 Comparison with human eye movements Validation using large collection of video clips Key defining characteristic of the work: do not attempt to develop models for high-priority content (e.g., human faces); instead, develop model of the neurobiology of human visual processing. CS 597
CS 597 Itti & Koch, Nature Reviews Neurosci 2001
Comparison to eye movements • 50 video clips, 0:06 to 1:30 each, over 25 minutes total • Outdoors day & night, parks, crowds, rooftop bar, etc • TV news, sports, commercials, talk shows, etc • Video games, test stimuli • Task: “follow main actors and actions, try to understand overall what happens in each clip. We will ask you a question about main contents. Do not worry about details like specific text messages.” • 8 subjects total, 4 to 6 subjects / clip (4.7 avg) • Normal corrected vision, ages 23-32, 3 females, 5 males, mixed ethnicities • ISCAN tracker • 240Hz recording, 9-point calibration every 5 clips • Affine + thin-plate-spline calibration transform [Bookstein, 1989] • 640x480 @ 60.27Hz doublescan, 33.185ms/movie frame CS 597
Outdoors scene (easy) CS 597
Video game CS 597
TV, news (worst clip) CS 597
50 clips 46,489 frames CS 597
4.7 subjects/clip 50 clips Average blur at human eye = 1/2 x average blur over entire display p<0.01 for every clip using continuous blur map, And for 48/50 clips using 3 discrete foveas 46,489 frames CS 597
MPEG1: (average) Foveated compressed file size = 51% unfoveated compressed file size MPEG4: (average) Foveated compressed file size = 46% unfoveated compressed file size 50 clips 46,489 frames CS 597
Conclusions • Bottom-up salience based on simple interacting center-surround operators may explain much more than we often expect. • Video encoding priority predicted by simple salience model highly significantly correlates with human eye fixations, under our specific task. • Significant compressed file size reductions were achieved by automatic foveation. CS 597
The paper • June 30, 2003: Submitted to IEEE Transactions on Image Processing • October 15, 2003: reviews received – 2 liked it a lot, one liked the idea but not the writing. Out of typical editorial verdicts: • Reject • Revise and resubmit / major revision required with re-review • Minor changes required, usually without re-review • Accept as is ours is “revise and resubmit” • October 22, 2003: revision submitted along with answers to reviews • January 20, 2004: 2 reviewers liked the changes and recommend publication as is, one still wants a minor change. Paper status now: “accepted pending minor revision” • January 22, 2004: second revision submitted with answers • January 24, 2004: paper officially accepted • January 28, 2004: final version of text, separate figures, separate bibliography, author bio, etc sent to publisher. • June 24, 2004: official acceptance date (what happened Jan-June??) • October, 2004: publication CS 597
The reviews: first a few points of general feedback Reviewer 1 Comments: I. REVIEW Please expand and give details in Section III. A. Suitability of topic 1. Is the topic appropriate for publication in these transactions? (X) Yes ( ) Perhaps ( ) No 2. Is the topic important to colleagues working in the field? (X) Yes ( ) Moderately So ( ) No (explain: ) B. Content 1. Is the paper technically sound? (X) Yes ( ) No (why not? ) 2. Is the coverage of the topic sufficiently comprehensive and balanced? (X) Yes ( ) Important information is missing or superficially treated. ( ) Treatment somewhat unbalanced, but not seriously so. ( ) Certain parts significantly overstresses. [etc etc…] CS 597
The review: then a written commentary III. DETAILED COMMENTS Please state why you rated the paper as you did in Sections I and II. If you have indicated that revisions are required, please give the author specific guidance regarding those revisions, differentiating between optional and mandatory changes. This is an interesting approach and it is quite well described. The methods are essentially sound. The one concern I have with the approach is the nature of the instructions given to the subjects during the eye movement experiment, namely to 'not focus on small details.' This may not _guarantee_ the (favorable) outcome of the comparison with the model but it sure stacks the deck in favor of the model. This is, in fact, acknowledged by the author on p. 12 where he states that 'humans often fixated small details (...) instead of the most salient actor...despite the instructions given'. In other words, even though the subjects were explicitly instructed to behave like the model they did not always do so! [etc etc…] CS 597
Let’s write an answer • Here we start with some general evaluation of the reviews: • Key points to address here: • Reviewers always want more details, figures, validations, tests, … • Editors always want a shorter paper overall CS 597
Is that a good answer? • It does make a convincing argument • It is polite • It answers the question well • It is detailed enough CS 597
Is that a good answer? • NO – explaining to the reviewer why s/he misinterpreted your writing or what you intended to write or how s/he is missing some general knowledge in the field is not at all what we are after. • If a reviewer makes a comment, no matter how irrelevant, besides the point, due to careless reading, etc. it may look to you, you must address it in the manuscript. • Remember: If one out of 3 reviewers made that comment, one out of 3 readers are likely to make it. • So, your answer should focus not on answering the question/comment, but on explaining how you have modified the paper such that this comment/question is addressed in the paper. CS 597
Explain how you modified the paper • Document exactly where you modified it so that reviewers • and editor can check it CS 597
Note: you do not have to agree and comply with every Reviewer request, but you must demonstrate evidence That you have done something about every request. CS 597
If the reviewer missed something that was explained In the paper, readers may miss it too. Always modify The paper to address that problem. CS 597
If you are going to not comply with a request, be sure To back your decision up with some serious data! CS 597
Some of the revisions… CS 597
Revising the text is easy, but how about figures/data? • When you submit a paper, be sure to make a full archive of: • All the source code of all programs used • All the raw data • All the processing steps in perl/shell scripts • All the result analysis steps in perl/shell/matlab scripts • Etc • Etc • And of course remember the first commandment: “thou shalt never manipulate data with your bare hands” • Most reviewers will ask for revisions, no matter how good a paper is. You must be ready to address them! CS 597