Thank you for agreeing to review a paper for MICCAI 2009. Your reviews have a direct and important impact on the quality of this meeting. Your reviews also help the medical imaging and computer/robotic assisted intervention community as a whole to improve the quality of its research. Please read through the rest of this document that provide details on what is expected of you as a member of the Scientific Reviewing Committee for MICCAI 2009. If you have any questions, please e-mail the program chairs (firstname.lastname@example.org
The deadline for completed reviews is April 24, 2009. The papers will be assigned to you by March 27, 2009. So you have almost a month and half to do these reviews, please do not leave them to the last few days of the deadline. The PC members have a lot of work to do after the reviews are in. Adhering to this deadline is therefore extremely important to us.
As soon as you get your reviewing assignment, please go through all the papers to make sure that (a) there is NO obvious conflict with you (e.g., it is your recent collaborator from a different institution) and (b) you are qualified to review the paper assigned. If these issues arise, please respond right away using the online review system. Contact us also if you find a paper that violates any of the paper submission guidelines.
We will once again be offering an author rebuttal process this year before the Program Committee meeting. PC members may also follow up with you to get clarifications on specific reviews, particularly for those with diverging reviews.
What to Look For
Look for what’s good or stimulating in the paper. Minor flaws can be corrected and shouldn’t be a reason to reject a paper. Each paper that is accepted should, however, be technically sound and make a substantial contribution to the field. Please familiarize yourself with the information in the Call for Submissions.
Blind reviewing is an essential part of MICCAI reviewing. Authors have been asked to take reasonable efforts to hide their identities, including not listing their names or affiliations and omitting acknowledgments. This information will of course be included in the published version. Reviewers should also make all efforts to keep their identity invisible to the authors. Don’t say, “you should have cited my paper from 2006!”
Please be specific and detailed in your reviews. In the discussion of related work and references, simply saying “this is well known” or “this has been common practice in the industry for years” is not sufficient: cite specific publications or public disclosures of techniques! The Explanation section is easily the most important of the review. Your discussion, sometimes more than your score, will help the PC members decide which papers to accept, so please be thorough. Your reviews will be returned to the authors, so you should include any specific feedback on ways the authors can improve their papers. For more suggestions on writing your reviews, read the section below on Writing Technical Reviews.
When You’re Done
When you have finished with your review, you should destroy any paper manuscript and/or supporting material you received. See the Ethics guidelines below.
Writing Technical Reviews
Here are some recommendations that may help you as you do this very valuable task.
In many professions, people give back to their community by doing volunteer work. In technical fields, we volunteer our time by reviewing papers that are written by other researchers in our field. We recommend that you approach your reviews in this spirit of volunteerism. Sure, your reviews make you a gatekeeper in helping decide which papers are ready for publication. Just as important, however, is to provide feedback to the authors so that they can improve their work. Try to write your review in a way that the authors can benefit from your review.
Most reviewers like reading a paper and then thinking about it over the course of several days before writing the review. “Living” with a paper for a few days gives you time to make thoughtful decisions about it. This is the best way to come up with helpful suggestions for improving the paper. To do this, you need to carve out some time in your day to think about the paper that you are reviewing.
The tone of your review is important. A harshly written review may be disregarded by the authors, regardless of whether your criticisms are constructive or not. If you take care, it is always possible to word your review diplomatically while staying true to your thoughts about the paper. Put yourself in the mindset of writing to someone you wish to help, such as a respected colleague who wants your opinion on a concept or a project.
Here are some specific issues to keep in mind as you write your reviews:
Short reviews are unhelpful to the authors and to other reviewers. If you have agreed to review a paper, you should take enough time to write a thoughtful and detailed review.
Be specific when you suggest that the writing needs to be improved. If there is a particular section that is unclear, point it out and give suggestions for how it can be clarified.
Don’t give away your identity by asking the authors to cite several of your own papers.
If you don’t think the paper is right for the MICCAI program, suggest other publication possibilities (journals, conferences, workshops) that would be a better match for the paper.
Avoid referring to the authors by using the phrase “you” or “the authors.” These phrases should be replaced by “the paper.” Directly talking about the authors can be perceived as being confrontational, even though you do not mean it this way.
Be generous about giving the authors new ideas for how they can improve their work. Your suggestions may be very specific (for example, “this numerical solver would be better for your application”) or may be more general in nature. You might suggest a new dataset that could be tried, or a new application area that might benefit from their tool. You may tell them how their idea can be generalized beyond what they have already considered.
A thoughtful review not only benefits the authors, but may well benefit you, too. Remember that your reviews are read by other reviewers and especially the PC members, in addition to the authors. Being a helpful reviewer will generate good will toward you in the research community.
Ethics for Reviewing Papers
1. Protect Ideas
As a reviewer for MICCAI, you have the responsibility to protect the confidentiality of the ideas represented in the papers you review. MICCAI submissions are by their very nature not published documents. The work is considered new or proprietary by the authors; otherwise they would not have submitted it.
Of course, their intent is to ultimately publish to the world, but most of the submitted papers will not appear in the MICCAI proceedings. Thus, it is likely that the paper you have in your hands will be refined further and submitted to some other journal or conference, or even to MICCAI next year. Sometimes the work is still considered confidential by the author’s employers. These organizations do not consider sending a paper to MICCAI for review to constitute a public disclosure. Protection of the ideas in the papers you receive means:
Do not show the paper to anyone else, including colleagues or students, unless you have asked them to write a review, or to help with your review.
Do not show any results or videos/images or any of the supplementary material to non-reviewers.
Do not use ideas from papers you review to develop new ones of your own before its publication.
After the review process, destroy all copies of papers and videos that are not returned to the senior reviewer and erase any implementations you have written to evaluate the ideas in the papers, as well as any results of those implementations.
2. Avoid Conflict of Interest
As a reviewer of a MICCAI paper, you have a certain power over the reviewing process. It is important for you to avoid any conflict of interest. Even though you would, of course, act impartially on any paper, there should be absolutely no question about the impartiality of review. Thus, if you are assigned a paper where your review would create a possible conflict of interest, you should return the paper and not submit a review. Conflicts of interest include (but are not limited to) situations in which:
You work at the same institution as one of the authors.
You have been directly involved in the work and will be receiving credit in some way. If you’re a member of the author’s thesis committee, and the paper is about his or her thesis work, then you were involved.
You suspect that others might see a conflict of interest in your involvement. For example, even though Microsoft Research in Seattle and Beijing are in some ways more distant than Berkeley and MIT, there is likely to be a perception that they are “both Microsoft,” so folks from one should not review papers from the other.
You have collaborated with one of the authors in the past three years (more or less).
Collaboration is usually defined as having written a paper or grant proposal together, although you should use your judgment.
You were the MS/PhD advisor of one of the authors or the MS/PhD advisee of one of the authors. Funding agencies typically consider advisees to represent a lifetime conflict of interest. MICCAI has traditionally been more flexible than this, but you should think carefully before reviewing a paper you know to be written by a former advisee.
The blind reviewing process will help hide the authorship of many papers, and senior reviewers will try hard to avoid conflicts. But if you recognize the work or the author and feel it could present a conflict of interest, send the paper back to the senior reviewer as soon as possible so he or she can find someone else to review it.
3. Be Serious
The paper publishing business in MICCAI is very serious indeed: careers and reputations hinge on publishing in the proceedings, academic tenure decisions are based on the proceedings, and patent infringement cases have discussed whether something was considered novel enough to publish in the proceedings. This does not mean that we cannot have any fun in the paper sessions. But it does mean that we have a responsibility to be serious in the reviewing process. You should make an effort to do a good review. This is obvious. But one of the complaints we have heard about the MICCAI review process is that some reviews can be so sketchy that it looks like the reviewer did not even seem to take the time to read the paper carefully. A casual or flippant review of a paper that the author has seriously submitted is not appropriate. In the long run, casual reviewing is a most damaging attack on the MICCAI conference. There is no dishonor in being too busy to do a good review, or to realize that you have over-committed yourself and cannot review all the papers you agreed to review. But it is a big mistake to take on too much, and then not back out early enough to allow recovery. If you cannot do a decent job, give the paper back and say so. But please, do it early so that the PC members and Program Chairs have time to select another reviewer before the deadline.
4. Be Professional
Belittling or sarcastic comments may help display one’s wit, but they are unnecessary in the reviewing process. The most valuable comments in a review are those that help the authors understand the shortcomings of their work and how they might improve it. If you intensely dislike a paper, justify it constructively and still provide feedback to the authors. If you give a paper a low score, it is essential that you justify the reason for the score in detail. Just saying “I do not like this approach because I have 10+ years of experience in this area” is NOT constructive. You need to share your professional opinion.
5. In Summary
Adherence to ethics makes the whole reviewing process more complicated and sometimes less efficient. But convenience, efficiency, and expediency are not good reasons to contravene ethics. It is precisely at those times when it would be easier or more efficient to bend the rules that it is most important to do the right thing. Ultimately, spending that energy and time is an investment in the long-term health of the technical-paper sessions, the conference, and our research community. Once again, thank you for all your support.
The above guidelines are based on Specific Documents Created for SIGGRAPH 2008 by Greg Turk and CVPR 2009 (used here with permission).