Guidelines for Referees

Please bear this information in mind when reviewing the proposal.

EVALUATION PROCESS
Referees may refuse to review a proposal if they consider it too dissimilar to their field of expertise or if a conflict of interest exists. A conflict of interest occurs when a person may benefit either professionally or personally from the success or failure of a proposal.
Should you not feel suitably qualified to perform this review, are directly involved with or have a vested interest in the proposal, or cannot meet the deadline, please contact the office immediately.

EVALUATION CRITERIA

Proposals must be evaluated according to the following evaluation criteria:

Scientific quality, scope and – where appropriate – interdisciplinarity of the proposal
The proposed conference should be a high-level conference and should satisfy a need in the scientific community. The topic should be at the forefront of scientific research and at the highest scientific level with respect to the selection of sessions and the choice of speakers. For interdisciplinary proposals, authors should describe how each discipline will be covered and how interaction between disciplines is to be achieved.

Scientific relevance in terms of novelty, originality, timeliness and contribution
The proposal should focus on a topic that is an acknowledged or emerging challenge for research. Novelty and innovation should be present either in the topic itself, or in the approach that is taken to discuss this topic (e.g. interdisciplinary, methodologically, empirically).

European dimension and added value to the international research community
The proposal should focus on a topic that is not just of local or regional importance, but that is relevant to researchers from different regions, countries, sectors and possibly even disciplines. It has to be evident that there is a need to discuss this topic in a European or international context.

Scientific quality of the programme
Through the programme, proposers should demonstrate that they can implement and achieve the ideas and objectives described in the conference proposal. The draft programme should reflect the aims and objectives of the IS International Conference Scheme.

Scoring and comments
You will be asked to score each evaluation criteria on a scale ranging from ‘average or less’ to ‘outstanding’. For each criterion under examination, score values indicate the following:
Average or Less There are serious inherent weaknesses in relation to the criterion in question
Good While the proposal broadly addresses the criterion, there are weaknesses that would need correcting
Very Good The proposal addresses the criterion well, although certain improvements are possible
Excellent The proposal successfully addresses all relevant aspects of the criterion in question
Outstanding/Top Priority  The proposal successfully addresses all relevant aspects of the criterion and is highly relevant and timely. The proposal demonstrates exceptional qualities and depth.

Maintain consistency in your scoring throughout the report. Projects will be ranked according to their merit.
You are also required to provide a comment for each criteria being assessed. You are encouraged to write your comments in a way that clearly reflects your overall opinions and specific strengths and weaknesses of the proposal for each criterion. The comments must be consistent with the score awarded.

Further Information on the Review and Rebuttal Process
In order to improve the process, referees have the opportunity to ask the convener specific questions related to the proposal. This may be necessary if the proposal is unclear or clarification of a specific point is required. Referees must use the email address to ask questions. Questions in any other part of the form will not be allowed. Authors will then be given the opportunity to respond during the rebuttal process.
The IS International Conference rebuttal process allows a proposer to respond to the reviews of his or her proposal. The rebuttal is for addressing factual errors in the reviews and for answering specific questions posed by. It is limited to 500 words of text per section, and must be self-contained (i.e. no URLs to external pages; no uploads). No revisions of or additions to the original proposal text will be accepted.

Best Practice Tips for International Peer Review
In order to support referees during this process and to ensure the standard of the reviews received, we have developed several best practice tips for writing an international peer review. Ideally, referees should consider all of the key points below.

Accuracy
Ensure that your review is technically accurate.

Assessment
Assess and mark the proposal exactly as it is described and presented. Do not make any assumptions or interpretations about the project in addition to what the proposers themselves have written. Keep to the evaluation criteria as described above. You should also consider whether the conveners substantiate the claims made in the proposal, whether the science is sound.

Productive
Where justified, give recommendations for modifications of the proposal. Reviews will be accessible to the proposers at the end of this process so please ensure comments are constructive. Avoid general statements such as “The objectives could have been better described”. Also avoid generalizations such as “Organization X is weak in this area”. Say rather “It has not been demonstrated in the proposal that the applicant has the capacity to run the project”.

Well-written
The review should be well written, organized, and free of typographical and spelling errors. Your comments should be concise, complete and comprehensible. You should use polite and correct language, but do not hide the facts.

Thank you and good work.