UAI 2013 - The Logistics of the Reviewing Process
Thank you if you are a reviewer for UAI 2013. The quality of the conference program is greatly dependent on the time and effort you expend in reviewing. Please carefully read the instructions and guidelines below. Note that all dates below should be interpreted as 11:59 UTC on the stated date.
Initial Registration (to be completed before March 1st)
1. Log in to the Conference System
Please make sure you can access the online Conference Management Tool (CMT) and that your contact information is correct.
2. Indicate your list of interests/expertise
Within CMT please access the "Subject Areas" function and please put a check beside any topic for which you would be willing to review papers. See Subject Areas for more information about the subject areas.
3. Register with the U Toronto paper-matching system
You can register here if you do not have an account yet.
Paper Bidding (to be completed between March 4th and March 11th)
Once papers have been submitted to the conference, you will be notified that the bidding process has started, and you will be allowed to bid on papers in the CMT system. Additional details on this process will be provided beforehand.
Writing Reviews (to be completed between March 15th and April 12th)
After the bidding process, you will be notified by email (around March 15th) about what papers you have been assigned and you will be able to access them online.
Note that each paper will also be assigned to a senior program committee (SPC) member who will act as the coordinator among reviewers for this paper, if discussion of a paper is needed. The SPC member for a paper is your primary point of contact at all times if you have any questions about the paper, e.g., if for any reason there is a problem with the paper (e.g., it exceeds the required page length) or some other problem such as a conflict of interest that was not detected earlier. If for any reason you are not able to review a paper, please let the SPC member know ASAP so an alternative reviewer can be found.
Please carefully read the guidelines for paper reviewing which provides guidance on both what the content of your reviews should contain as well as information about the numerical scoring system being used for reviews.
Below are the dates by which various items need to be completed - please note that these are hard deadlines:
- Friday April 12th: for all of your papers, complete your written reviews and assign numerical scores.
- Monday April 15th to Friday April 26th: for each paper, if the responsible SPC member determines that it requires discussion, then you will discuss the paper by email with the other reviewers and can revise your written review and/or scores any time up until April 26th. More information on the discussion process is provided below.
Discussion Period (April 15th to 26th)
After reviews are submitted by April 12th, the SPC members will look at the reviews for each paper that they are responsible for and decide if a discussion among the reviewers is needed. The purpose of the discussion is to allow the reviewers and SPC member to either (a) resolve issues where there is significant disagreement about a particular paper among reviewers (e.g., a combination of some very high and very low scores), or possibly (b) where a paper is right on the borderline of acceptance and merits further discussion. For any paper that goes into "discussion mode" you will be able to see the reviews of the other reviewers for this paper. The SPC member will moderate the discussion and may ask particular reviewers to elaborate on particular aspects of their review.
If a reviewer so wishes, he or she can update their written review and numeric scores (no later than April 26th) based on the discussion with the other reviewers and the SPC member.
Note that the purpose of the discussion period is not to force all reviewers to agree on a common score - if there is a valid difference of opinion, this is fine, as long as each individual review makes it clear what the basis is for the stated opinion and scores.
Final Decision Period (April 26th to May 6th)
After April 26th the reviewers have officially completed their work (thanks in advance!). The SPC members will provide a meta-review to summarize the discussion (if they believe that this is needed) by April 26th. The PC Chairs (in consultation with the SPC members) will then make final accept/reject decisions, with announcements to all authors by Monday May 6th.