Frequently Asked Questions

21, March: Frequently Asked Questions about Author-Response Phase section has been released. ( Direct link to FAQ)
Frequently Asked Questions about Being an SPC section has been released. ( Direct link to FAQ)
Frequently Asked Questions about Reviewing Papers section has been released. ( Direct link to FAQ)
Frequently Asked Questions about summary-reject phase section has been released.
Frequently Asked Questions about Submissions section has been released.

Frequently Asked Questions about Response Phase

Frequently Asked Questions about Being an SPC

Frequently Asked Questions about Reviewing Papers

1. What is the summary-reject phase?

The idea is inspired from crowdsourcing. Several people work a little
to achieve the task that would require significant work from a single
person. You will be between 7 and 10 SPCs to have a very quick look
(five to ten minutes) at a paper and vote whether yes or no it
deserves to go for standard “full” review process (three PCs writing
reviews and one SPC monitoring). The area chair in charge of the paper
will analyse the votes and will take the decision on whether or not to
keep this paper or discard it.

2.Why a phase of summary reject?

IJCAI faces a tremendous increase of submissions. If we only rely on
the standard review process, we need a number of people that is above
the number of qualified people we can recruit. The consequence is a
drop in the quality of the reviews (either because of the load per
reviewer or because of unqualified ones). The purpose of the
summary-reject phase is to significantly decrease the number of papers
going for full review by discarding papers that clearly do not reach
the criteria of quality expected from an IJCAI paper. The summary
reject phase is performed by senior researchers who can quickly assess
the approximate level of a submission, or at least quickly notice that
a paper will not make it.

3.What is the level of quality expected from an IJCAI paper?

You are senior people so, if you are from classic AI areas, with a
long tradition of publication at IJCAI, you know that people submit
their *best* papers. For areas which have grown a lot in the last few
years (NLP, vision, machine learning), it is less evident. A rule of
thumb to decide if a paper has the IJCAI standard of quality is to ask
yourself whether the paper is good enough for ACL/ICCV/ICML. If not,
it is not good enough for IJCAI either.

4.How should I spend my few minutes on a paper?

In five to ten minutes you obviously cannot evaluate the algorithms,
the theorems or the tables of results. Read first the abstract. It
often tells you quickly that this work is weak or not exciting at
all. If doubts, jump to the conclusion, which summarizes the
contribution. The structure of the body itself can also show you an
evidence of lack of structure, lack of contribution, etc.

5.How should I decide whether a paper should go for full review process or should be summary rejected?

Any paper that has clear weaknesses that we do not expect from IJCAI
papers should be summary rejected. Examples of weaknesses can be:
– too badly written/organized
– not motivated
– not positioned wrt related approaches
– too incremental
– not evaluated (depending on the contribution, it could be
experimental, theoretical, etc)
– no technical contribution, just an application/tuning of existing techniques
(except if the problem it solves has a clearly demonstrated high impact on society)
– clearly not belonging to the area declared by the author(s)

6.What if I am scared to make a mistake in my vote?

I know that in most cases a few minutes is not enough to be totally
sure of your vote. To reduce your fear to make a mistake, bear in mind
that you will not be the only SPC to vote on the paper. You will be
from 7 to 10 depending on the area. Thus, the decision to summary
reject the paper is not in your own hands only. This means that you do
not need to be totally sure a paper is weak to vote NO. The paper will
be summary rejected only if the trend of votes in NO and the AC in
charge of the paper agrees with these votes.

7.How to vote in CMT?

Once you are on the page displaying the papers assigned to you (check
that your role is “meta reviewer”, not “author”), you will see that
for each paper, in the column called “Entered note?”, it is written
“No”, followed by a button “Add Note”. Click on it. You will arrive on
the voting page where you have three questions:

+ The vote itself (YES/NO) depending on whether you think the paper
should undergo a full review process or not.
+ Your confidence in your vote (High/Moderate/Low).
+ A window for an optional comment supporting your vote. This will
help the area chair in taking a decision in borderline cases.

Then press “Submit” at the bottom of the page. It comes back to the
pages with your papers. The cell in the third column is now “Yes, Edit
Note” instead of “No, Add Note”. You can later come back and edit your
note if needed.

***Warning***: This “Yes” in the third column does not mean you voted
Yes. It just means you have voted!

Frequently Asked Questions about Submissions

1. Can I submit a paper to IJCAI if it is already on arXiv?

Yes. We require that any submission to IJCAI must not be already
published or under review at another archival conference or journal.
Papers put on arXiv do not violate this rule as long as
they are not cited by the submitted paper.

2. A preliminary version of my paper has already been presented at a
workshop or a conference without formal proceedings. It it ok to
submit it to IJCAI?

Yes. This applies more generally to papers presented in any
non-archival venue (that is, as long as there are no formally
published proceedings with a publisher).

3. How strict is the author anonymity requirement in the submissions?

IJCAI requires that all submissions be anonymised. To facilitate this,
we require that the authors follow both the letter and spirit of the
normal anonymisation guidelines. Any paper that obviously violates
this requirement will be rejected without review.

4. Can I submit the title and abstract of paper which is under review
for a conference whose notification date is between January 16 and
January 21?

No.

5. My paper has been rejected from AAAI 2020, ECAI 2020, AAMAS 2020, ICAPS 2020.
Do I have to attach the reviews and the response letter?

Yes. Resubmissions from these conferences have to follow the
“Resubmissions of Substantially Improved Recent Submissions” in the call for papers.

6. Is supplementary material allowed for IJCAI submissions?
Are anonymized links to supplementary material allowed?

No. Neither supplementary material nor anonymized links are allowed.
If a paper goes over the page limit or includes any link to
supplementary material on the web, it will be automatically rejected without review.
If extra material, such as full proofs, additional figures or examples,
or more complete data sets are explicitly asked for by reviewers,
such a link may be provided by the authors during the response phase.
This scenario will be used in exceptional cases only and reviewers are
not required to look at this supplementary material.
Submissions to IJCAI-PRICAI should be self-contained.

7. When does a paper qualify as a student paper?

A paper is considered a ‘student paper’, and thus a potential winner
of the ‘Best Student Paper Award’, when the first author is a student
who, at the time of submission, has not obtained any PhD degree
yet. When submitting a paper, this should be indicated by marking the
appropriate checkbox.

8. Can I submit a paper consisting of an extension of a previously
published short paper / extended abstract?

Yes if, for instance, the short paper is a two-column two-page
paper. If the ‘short’ paper is longer than that, but no longer than
half of the IJCAI paper, then the extension should be significant
enough. Significance will be assessed by reviewers. In any case, the
short version has to be cited and it must be clear that the
IJCAI-PRICAI submission is an extension.

9. Is there a maximal length for titles and abstracts to be submitted
on the website?

Title: It is highly recommended not to go above two lines in the IJCAI
format. Three lines is a strict maximum.

Abstract:
There is no word limit for abstracts. However, 200 words is a nice
upper limit that authors should always apply.

10. What does ‘abstract submission’ mean? Do I have to submit an
abstract if I intend to submit a full paper? Can I submit an abstract
only?

By ‘abstract submission’ we mean that you should register your paper
by giving a title and a short abstract (see FAQ 9). This abstract is
intended for helping program committee members selecting the papers
they are competent for reviewing. There are no abstract-only
submissions, nor any paper-only submissions: in all cases, you must
submit an abstract, and then you must submit a full paper. (See
abstract submission and paper submission deadlines on the web site).

11. When submitting a paper I have to agree with the IJCAI-PRICAI 2020
submission policy, which says that no additional author will be
allowed after the paper submission deadline has passed. Will I be
allowed to change the author order if the paper is accepted?

Yes.