UAI 2015 - The Logistics of the Reviewing Process

Thank you if you are a reviewer for UAI 2015. The quality of the conference program is greatly dependent on the time and effort you expend in reviewing. Please carefully read the instructions and guidelines below. Note that all dates below should be interpreted as 11:59pm UTC on the stated date.

Initial Registration

1. Log in to the Conference System

Please make sure you can access the online Conference Management Tool (CMT) and that your contact information is correct.

2. Indicate your list of interests/expertise

Within CMT please access the "Subject Areas" function and please put a check beside any topic for which you would be willing to review papers. See Subject Areas for more information about the subject areas.

Paper Bidding
(March 4th - March 9th)

Once papers have been submitted to the conference, you will be notified that the bidding process has started, and you will be allowed to bid on papers in the CMT system. Additional details on this process will be provided beforehand.

Reviewer Assignment by SPC members
(March 11th - March 17th)

SPC paper assignment will be announced on March 10th. Two reviewers (PC members) per paper will be assigned more or less automatically, based on bids, topic similarity, and TPMS (Toronto Paper Matching System) scores. SPC members will be asked to pick the third reviewer. This is important for finding a good match between the papers and the reviewers, and hence conducive to high quality reviewing.

Writing Reviews by PC Members
(March 18th - April 12th)

PC paper assignment will be announced on March 18th. Each paper will have 3 reviewers (PC members) and 1 meta-reviewer (SPC member).

Please carefully read the guidelines for paper reviewing which provides guidance on what the content of your reviews should contain as well as information about the numerical scoring system being used for reviews.

The reviews are due on April 12th. Please note that it is a hard deadline.

Discussion Period
(April 24th - May 4th)

Authors will be invited to provide feedback on the initial reviews by April 23rd. Thereafter, the meta-reviewers will look at the reviews for each paper that they are responsible for and decide if a discussion among the reviewers is needed.

The purpose of the discussion is to allow the reviewers and meta-reviewer to either (a) resolve issues where there is significant disagreement about a particular paper among reviewers (e.g., a combination of some very high and very low scores), or possibly (b) where a paper is right on the borderline of acceptance and merits further discussion. For any paper that goes into "discussion mode" you will be able to see the reviews of the other reviewers for this paper. The meta-reviewer will moderate the discussion and may ask particular reviewers to elaborate on particular aspects of their review.

If a reviewer so wishes, he or she can update their written review and numeric scores (no later than May 4th) based on the discussion with the other reviewers and the meta-reviewer.

Note that the purpose of the discussion period is not to force all reviewers to agree on a common score - if there is a valid difference of opinion, this is fine, as long as each individual review makes it clear what the basis is for the stated opinion and scores.

The SPC members will provide a meta-review on each paper by May 4th to summarize the reviews and discussions, and to make recommendations regarding the accept/reject and plenary/poster decisions.

Final Decision Period
(May 4th - 11th)

After May 4th, the reviewers have officially completed their work (thanks in advance!). The PC Chairs (in consultation with the SPC members) will then make final accept/reject decisions, with announcements to all authors by Tuesday May 12th.