| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| reviewer_instructions [2011/02/02 10:25] – nickroy | reviewer_instructions [2011/03/15 22:38] (current) – [Detailed Review Instructions] nickroy |
|---|
| [[:home|Home]] | [[:home|Home]] |
| |
| <fs 2em>RSS 2011 Reviewer Instructions</fs> | #####RSS 2011 Reviewer Instructions##### |
| |
| ----- | ----- |
| * Jan 30 Reviews are assigned\\ | * Jan 30 Reviews are assigned\\ |
| * Feb 28 Reviews due \\ | * Feb 28 Reviews due \\ |
| * Mar 7 Author rebuttal begins\\ | * Mar 7 Author response begins\\ |
| * Mar 14 Author rebuttal ends, reviewers answer to rebuttals, discussion begins \\ | * Mar 14 Author response ends, discussion begins, reviewers revise initial reviews \\ |
| * Mar 28 Discussion period ends\\ | * Mar 28 Discussion period ends\\ |
| * Apr 7 Area chair meeting \\ | * Apr 7 Area chair meeting \\ |
| * Jun 27 RSS in Los Angeles | * Jun 27 RSS in Los Angeles |
| |
| Please be guided by *this* schedule, not the deadlines listed by CMT. All CMT deadlines are listed as 4/15. The CMT deadline is the time at which CMT turns off access to parts of the system. In the course of the discussion and the acceptance decisions, the area chairs may ask your (the reviewer) to clarify a point in your review. This cannot happen after the CMT deadline. To allow revisions to rebuttals or the discussion, we put the CMT deadlines once the acceptances are announced. | Please be guided by __**this**__ schedule, not the deadlines listed by CMT. All CMT deadlines are listed as 12/25/25. The CMT deadline is the time at which CMT turns off access to parts of the system. In the course of the discussion and the acceptance decisions, the area chairs may ask your (the reviewer) to clarify a point in your review. This cannot happen after the CMT deadline. To allow revisions to rebuttals or the discussion, we put the CMT deadlines to a point substantially in the future after the acceptances are announced. |
| |
| ----- | ----- |
| * Once logged in, you should be presented with a webpage that looks very much like this image (click image to enlarge) : | * Once logged in, you should be presented with a webpage that looks very much like this image (click image to enlarge) : |
| |
| {{:public_images:reviewer_desktop.png?200x49&direct}}. | {{gallery>:public_images:reviewer_desktop.png?200x49&lightbox}} |
| |
| * Click on "Download Submission Files for All Assigned Papers" to get a zip file of all your assigned papers. | * Click on "Download Submission Files for All Assigned Papers" to get a zip file of all your assigned papers. |
| * Click on the appropriate paper, and you should be presented with a webpage that looks very much like this image (click image to enlarge): | * Click on the appropriate paper, and you should be presented with a webpage that looks very much like this image (click image to enlarge): |
| |
| {{:public_images:reviewer-panel.png?200x50&direct}}. | {{gallery>:public_images:reviewer-panel.png?200x50&lightbox}} |
| |
| * The ranking scores will not be available during Feedback, only the written comments. The red text <html><span style="color:red;">Visible To Authors Only After Decision Notification, Visible To Authors During Feedback</span></html> is contradictory. It should say "Visible To Authors Both During Feedback and After Decision Notification". (We were unable to get this corrected by CMT.) | * The ranking scores will not be available during Feedback, only the written comments. The red text <html><span style="color:red;">Visible To Authors Only After Decision Notification, Visible To Authors During Feedback</span></html> is contradictory. It should say "Visible To Authors Both During Feedback and After Decision Notification". (We were unable to get this corrected by CMT.) |
| * **Less helpful: "This work contains little or no experimental results."** While robotics is to a great extent the marriage of good experimentation and good theory, we welcome papers that provide insight into robotics through novel theoretical or algorithmic contributions. If you are unconvinced that the theory is correct, then a more helpful comment would be: "This is a theoretical paper, but unfortunately the authors make the following assumptions XX, YY and ZZ that undermine confidence that this theory can be applied usefully. We would need to see either experimental evidence, or a relaxation of those assumptions to be convinced of the authors' contribution." Or perhaps if the theoretical claim is uninteresting, a more helpful comment might be: "The authors provide a proof to support the claim of XX, but this claim is somewhat unsurprising given the work in YY and ZZ. A more interesting claim might be QQ, but the proofs do not support that result." | * **Less helpful: "This work contains little or no experimental results."** While robotics is to a great extent the marriage of good experimentation and good theory, we welcome papers that provide insight into robotics through novel theoretical or algorithmic contributions. If you are unconvinced that the theory is correct, then a more helpful comment would be: "This is a theoretical paper, but unfortunately the authors make the following assumptions XX, YY and ZZ that undermine confidence that this theory can be applied usefully. We would need to see either experimental evidence, or a relaxation of those assumptions to be convinced of the authors' contribution." Or perhaps if the theoretical claim is uninteresting, a more helpful comment might be: "The authors provide a proof to support the claim of XX, but this claim is somewhat unsurprising given the work in YY and ZZ. A more interesting claim might be QQ, but the proofs do not support that result." |
| * **Less helpful: "This is just a systems paper."** Again, while robotics is to a great extent the marriage of good theory and good experimentation, we welcome papers that provide insight into robotics through system development. This probably means that it is an experimental paper, but you are unconvinced by the experimentation, or even the claim of the paper. A more helpful comment would be: "This is an experimental paper, but unfortunately the experiments do not give sufficient confidence in the insight the authors describe. We would need to see the following experiments to be convinced of the authors' contribution." Or perhaps if the experimental claim is itself unclear or uninteresting, a more helpful comment might be: "The authors provide experiments but do not give a clear hypothesis being tested by XX. The results are somewhat unsurprising given the work in YY and ZZ. A more interesting claim might be QQ, but the experiments do not support that result." | * **Less helpful: "This is just a systems paper."** Again, while robotics is to a great extent the marriage of good theory and good experimentation, we welcome papers that provide insight into robotics through system development. This probably means that it is an experimental paper, but you are unconvinced by the experimentation, or even the claim of the paper. A more helpful comment would be: "This is an experimental paper, but unfortunately the experiments do not give sufficient confidence in the insight the authors describe. We would need to see the following experiments to be convinced of the authors' contribution." Or perhaps if the experimental claim is itself unclear or uninteresting, a more helpful comment might be: "The authors provide experiments but do not give a clear hypothesis being tested by XX. The results are somewhat unsurprising given the work in YY and ZZ. A more interesting claim might be QQ, but the experiments do not support that result." |
| | * **Less helpful: "This paper is basically uninteresting."** This comment is not helpful to the author. Even if you, as a reviewer, are tired of reading papers in a particular area, this comment gives no guidance to the authors. Please avoid comments of this form. It's possible to say that a paper is narrow in the problem it addresses, without actually calling it boring. Describing a problem as narrow encourages the authors to broaden the scope of the paper. Describing a paper as boring encourages the authors to set the building on fire out of frustration. |
| |
| Guidance on the meaning of the criteria and their numerical scores (Thanks to Dieter Fox). | Guidance on the meaning of the criteria and their numerical scores (Thanks to Dieter Fox). |