Guide for Reviewers

Guide for Reviewers#

Welcome to the guide for notebooks reviewers of the EDS book!

Before you begin this process, please be sure to understand how the publication process works.

We have a code of conduct which is mandatory for everyone involved in the review process of our notebooks.

Below you will find the main steps we suggest to follow to review notebooks submitted to EDS book.

Scope and Aims#

Thank you for considering reviewing a notebook for EDS book. Our review process aims to be open, collaborative, transparent and inclusive. We therefore welcome reviewers from a diversity of background and with varying levels of programming/domain expertise.

We encourage reviewers to adhere to the principles of the Open Science Peer Review Oath.

Pre-review#

An editor will confirm reviewers interest to review the notebook by tagging them into a PRE-REVIEW issue within the main EDS book repository. The PRE-REVIEW contains some relevant information of the notebook including a link to the notebook idea issue.

Review#

After confirming reviewers availability, Editors-in-Chief (EiC) opens a REVIEW issue to moderate a high-level discussion between notebook authors and reviewers.

Important

Reviewers will be asked to conduct the review within 2 weeks of accepting the invitation by the assigned editor. Authors will be asked to respond to reviewers’ comments within 2 weeks of the last-submitted review. We suggest an additional 1 week for reviewer approval of changes. Please contact the editor directly or in the REVIEW issue thread to inform possible delays.

To review a notebook, we provide a review template (see below) with a checklist.

Note

The checklist template follows the suggested review criteria for SciPy Proceedings and Turing Data Stories

*Please check off boxes as applicable, and elaborate in comments below. 
Your review is not limited to these topics, as described in the reviewer guide*

### Conflict of interest
- [ ] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor _before_ starting your review).

### Code of conduct an peer-review principles
- [ ] I confirm that I read and will adhere to the Environmental Data Science [code of conduct](https://github.com/alan-turing-institute/environmental-ds-book/blob/master/CODE_OF_CONDUCT.md).
- [ ] I confirm that I read and will adhere to [Open Science Peer Review Oath](https://doi.org/10.12688/f1000research.5686.2) 

### General checks
- [ ] **Notebook:** Is the notebook file part of the PR?
- [ ] **Contribution and authorship:** Does the author list seem appropriate and complete (full name, affiliation, and GitHub/ORCID handle)?
- [ ] **Scope and eligibility:** Does the submission contain an original and complete analysis according to the [theme](pb-about-aimsscope) selected?

### Reproducibility
- [ ] Does the notebook run in a local environment?
- [ ] Does the notebook build and run in binder?
- [ ] Are all data sources openly accessible and properly cited (e.g. with citation to a persistent DOI) in the heading section?

### Pedagogy
- [ ] Are the notebook purpose and highlights clear?
- [ ] Does the notebook demonstrate some specific data analysis or visualisation techniques?
- [ ] Is the notebook well documented, using both markdown cells and comments in code cells?
- [ ] Does the conclusion section provide clear and concise final say on the tools, analysis and/or datasets used?
- [ ] Is the notebook narrative well written (it does not require editing for structure, language, or writing quality)?

### Ethical
- [ ] Is any linkage of datasets in the notebook unlikely to lead to an increased risk of the personal identification of individuals?
- [ ] Is the notebook truthful and clear about any limitations of the analysis (and potential biases in data and/or tools)?
- [ ] Is the notebook unlikely to lead to negative social outcomes, such as (but not limited to) increasing discrimination or injustice?

### Other Requirements
- [ ] All mentioned software should be formally and consistently cited wherever possible.
- [ ] Acronyms should be spelled out upon first mention.
- [ ] Licence conditions on images and figures must be respected (Creative Commons, etc.).

### Final approval (post-review)

- [ ] **Authors has responded to my review and made changes to my satisfaction. I recommend approving the notebook for publication.**

The checklist will be available per reviewer within the first comment of the notebook REVIEW issue created by EiC.

Additional comments are welcome on the same issue or directly to the notebook repo.

If reviewers do the latter, they will find a PR in the notebook repository where authors and reviewers carry out the discussion through ReviewNB, a third-party plugin in GitHub for displaying and commenting Jupyter Notebooks (see further details here). Reviewers can open issues in the notebook repo too and link the URL of the notebook REVIEW issue thread. This facilitates centralizing comments. All changes implemented by authors will be available in the review PR. The editor will ask reviewers to re-evaluate the notebook after the authors’ changes.

Authors should respond within 2 weeks with their changes to the notebook in response to reviewers comments.