[Review]: metagenomics

[nselem]

Thank you for your interest in submitting your lesson for review in The Carpentries Lab!
Please respond to the prompts below to complete your submission.
Check boxes by adding an 'x' between the square brackets at the start of each point,
or submit the issue and check off the boxes afterwards.

---

1. What is the title of the lesson?
   Metagenomics Workshop Overview

2. Provide URLs to

   • The lesson repository:
     https://github.com/carpentries-incubator/metagenomics-workshop

   • The lesson homepage:
     https://carpentries-incubator.github.io/metagenomics-workshop/

3. Briefly describe the lesson (50 words or fewer).
   What does it aim to teach and to whom?
   This workshop teaches data management and analysis for metagenomics research, including best practices for organizing bioinformatics projects and data and connecting to and using cloud computing. It also provides experience using command-line utilities and tools to analyze sequence quality and R studio and R libraries to compare diversity between samples.

4. If you are submitting this lesson for review on behalf
   of multiple authors, list the GitHub usernames below for
   all authors who should receive notifications relating to the review.
   AbrahamAvelar, aaronejaime, fabel134, Vanessaarfer, Czirion, Bedxxe, nselem, BwanyaBrian, EdderDaniel, ahmedmoustafa

5. Provide URLs to workshop webpages and/or an issue
   on the lesson repository from any beta pilots of the lesson.
   (A beta pilot is a workshop where the lesson was taught
   by any instructor who was not part of the lesson development team
   before the pilot took place.)
   Alpha workshop https://betterlabmx.github.io/2020-11-27-BetterLab/
   First beta pilot https://czirion.github.io/2021-06-30-BetterLab-online/
   The second beta pilot was a course in a University, Here we provide the letter of the University to the professor: Guanajuato University Letter

6. (Optional) If you have obtained a DOI for the lesson via Zenodo,
   paste that DOI below.
   10.5281/zenodo.4285901

7. If the lesson is similar in topic to any other lesson
   already included in The Carpentries Lab and/or
   The Carpentries Lesson Programs (Software, Library, and Data Carpentry), briefly describe how this lesson differs and why a separate lesson was developed.
   This workshop is a curriculum that comprises four lessons. The first two lessons are adapted to metagenomics from the Genomics’ Data Carpentry. The third part includes a brief introduction to R, and the fourth lesson teaches a complete shotgun metagenomics workflow using public data which was not previously included in The Carpentrie’s lessons but it is a topic of interest to the biological community.

8. Check the boxes to confirm that the lesson

   • is the the original work of the author(s),
     or that any content derived from another source is reused with permission and appropriate attribution.
   • aligns with The Carpentries Code of Conduct.
   • is published under a CC-BY or CC0 license.
   • uses The Carpentries lesson template without significant customisation/adaptation.

9. If you wish to submit the lesson for publication in
   the Journal of Open Source Education (JOSE):
   (see the repository README for more details):

   • check the box to confirm that your repository includes
     paper.md and paper.bib files as described in
     the JOSE submission guide for learning modules.

[tobyhodges]

Thanks for submitting this lesson to The Carpentries Lab, @AbrahamAvelar, @aaronejaime, @fabel134, @Vanessaarfer, @Czirion, @Bedxxe, @nselem, @EdderDaniel, and @ahmedmoustafa.

(@BwanyaBrian doesn't seem to be a valid GitHub ID - @nselem please can you check and reply with the correct handle?)

I'll be acting as Editor on this submission, and I aim to work through the Editor checklist before the end of the work week. You can expect further posts to this thread after I have finished that.

For now, to ensure that the review process runs as smoothly as possible, please make sure you are subscribed to receive notifications from this thread. On the right sidebar of this page 👉 you should see a section headed Notifications, with a Customize link. You can click on that and make sure that you have the Subscribed option selected, to receive all notifications from the thread.

I will open Pull Requests on each of the lesson repositories, to add a badge tracking the status of your submission to the README.md.

Finally, for ease of future reference, I include here a list of links to all the lessons in this curriculum, which will be considered together in this review thread:

1. Metagenomics workshop overview
2. Project organization and management
3. Introduction to the command line
4. Introduction to R for Metagenomics
5. Data processing and visualization for metagenomics

[nselem]

Thank you for your revision Toby
The right user is @bwanya

[tobyhodges]

Performing the editorial checks on five lesson websites is taking a little longer than I anticipated. I am almost finished, but the working day is already over here and I won't be able to post my responses today. I am confident I will be able to add them here on Monday. Thanks for your patience.

[tobyhodges]

I've completed the editor checks on all five lessons. Taken together, the responses are quite long so I will post one for each lesson as a separate replies in this thread.

Please take some time to read through these responses, and post below if you have any questions about anything I've written. After you have addressed the points raised, please let me know by replying in this thread. When everything in my responses has been addressed, I can begin looking for reviewers for the curriculum.

[tobyhodges]

Editor checks - Metagenomics Workshop Overview

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

Images in Extras/Launching you own AMI instances are missing alternative text.
This is a page copied over from the Data Carpentry Genomics Workshop Overview site,
see my note about these Extras pages in the Content section, below.

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.
  • no “jumps” are present between heading levels e.g. h2->h4.
  • no page contains more than one h1 element i.e. none of the source files include first-level headings.
• The contrast ratio of text in all figures is at least 4.5:1.

There are some low contrast images in Extras/Launching you own AMI instances.
As mentioned above, this page is likely to be removed, in which case this comment can be ignored.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.
• All exercises have solutions. not applicable
• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.) not applicable
• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

The data on Zenodo is currently licensed CC-BY, and we would like lessons in The Carpentries Lab to use data in the public domain i.e. CC0. CC-BY and data: Not always a good fit gives a good summary of the problems with using the CC-BY license for data, and links out to some other helpful resources (the Dryad and BioMedCentral pages in particular).

I think there are two options to explore:

1. When you added the data to Zenodo, did you choose the CC-BY license for it then? If so, can you now adjust it to CC0?
2. Or was the data published under a CC-BY license by the original authors? If so, can you contact them and ask if they would be happy to release it into the public domain by making it CC0?

Note about content reproduced from DC Genomics curriculum

• Before the lessons can be reviewed, several of the pages under Extras should either be removed, or replaced with content more relevant to the Metagenomics curriculum.
• The Setup page also needs to be updated, to reflect the setup and data required for the Metagenomics curriculum rather than DC Genomics.

Design

• Learning objectives are defined for the lesson and every episode. not applicable
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.

The repository does contain a CC-BY license, but the copyright statement inside should be updated to remove Software and Data Carpentry. I suggest replacing it with:

... your work is derived from work that is Copyright © [authors' names] and, where practical, linking to https://carpentries-incubator.github.io/metagenomics-workshop/), ...

and

... the example programs and other software provided in the lesson are made available under the ...

The copyright statement in the footer of the lesson pages themselves should also be adjusted, but that would be taken care of by changing the lesson styling to use The Carpentries Incubator template instead of the Data Carpentry template being used now. I have opened pull requests to introduce these changes.

• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.

• a list of lesson maintainers.

Please add a list of the maintainers of the lesson to the repository README.md

• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises. not applicable
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson. not applicable

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.
• setup and installation instructions.

These are present but the page needs further modification, to remove instructions specific to the Data Carpentry Genomics lesson from which they are derived.

• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

This is missing but perhaps not required for an overview lesson site like this.

General

• To comply fully with the CC-BY license, this and the other lessons based on an equivalent from the DC Genomics curriculum should include a link to the original. I recommend something in index.md stating that the lesson is adapted from another, with the name of the original DC lesson and a link to the lesson website.
• The styling on the lessons should be consistent, and use The Carpentries Incubator template. I have opened pull requests on all five lesson repositories that, when merged, will take care of this.
• Please adjust the contact email address (email in the _config.yml file) to point to someone who people can contact about the curriculum.

[tobyhodges]

Editor checks - Project organization and management

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

Two images in 03-ncbi-sra need alternative text.
Here are some resources with advice on how to write good alt-text for images:

1. UCSF: Accessibile Images Best Practices
2. The bIg Hack: Avoid these common alt-text mistakes

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.
  • no “jumps” are present between heading levels e.g. h2->h4.
  • no page contains more than one h1 element i.e. none of the source files include first-level headings.

There are a few places where heading levels are skipped within the page content,
e.g. in 03-ncbi-sra, "Where to learn more" (h2) jumps to h4 ("About the Sequence Read Archive") instead of the next level down, h3.
The heading levels should be adjusted to avoid jumps like this between levels.

Several episodes include h1 headings in the page content.
These should be adjusted to h2 heading level, e.g. ## Accessing the original archived data instead of # Accessing the original archived data.

• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.

• All exercises have solutions.

• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)

• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

See my note about CC0 vs CC-BY in the response for the Workshop Overview lesson.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.

See my note about the copyright statements in the license file in the response for the Workshop Overview lesson.

• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.

• a list of lesson maintainers.

Please add a list of the maintainers of the lesson to the repository README.md

• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.

• setup and installation instructions.

See my note about the setup instructions in the response for the Workshop Overview lesson.

• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

Note about content reproduced from DC Genomics curriculum

• Before the lessons can be reviewed, several of the pages under Extras should either be removed, or replaced with content more relevant to the Metagenomics curriculum. I think it is ok to re-use the Instructor Notes from the original DC Genomics lesson here, because the two lessons are so similar, but I would like you to add a note to the top of that page that acknowledges that these notes were written for another lesson and that the Instructor might expect their experience to differ for a Metagenomics workshop because of that.

General

• Please adjust the contact email address (email in the _config.yml file) to point to someone who people can contact about the curriculum.

[tobyhodges]

Editor checks - Introduction to the Command Line for Metagenomics

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

Alt-text is present for all four images in the lesson but is currently not descriptive and can be improved.

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.
  • no “jumps” are present between heading levels e.g. h2->h4.
  • no page contains more than one h1 element i.e. none of the source files include first-level headings.

06-organization contains a few h1 headings in the page content. The heading levels in this page should be adjusted down.

• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.

• All exercises have solutions.

• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)

• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

See my note about CC0 vs CC-BY in the response for the Workshop Overview lesson.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• [] a CC-BY or CC0 license.

See my note about the copyright statements in the license file in the response for the Workshop Overview lesson.

• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.

• a list of lesson maintainers.

Please add a list of the maintainers of the lesson to the repository README.md.
The README for this lesson needs to be updated to distinguish the repository from the DC Genomics equivalent,
e.g. by changing the title and removing the citation information.

• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.
• setup and installation instructions.

See my note about the setup instructions in the response for the Workshop Overview lesson.

• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

General

• I noticed that 05-writing-scripts includes a broken link, to the non-existent page "https://carpentries-incubator.github.io/metagenomics/05-workflow/index.html"
• Please adjust the contact email address (email in the _config.yml file) to point to someone who people can contact about the curriculum.

[tobyhodges]

Editor checks - Introduction to R for Metagenomics

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

Please add alternative text for the images in this lesson.

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.

The highest heading level in most episodes is h3. The heading levels in these episodes should be adjusted so that sections begin with h2 headings.

• no “jumps” are present between heading levels e.g. h2->h4.

h6 headings are used for image captions. If you want to use captions, please replace these elements with something else, e.g. italicised text. The caption text you have chosen could be a good fit as alt-text on the images.

• no page contains more than one h1 element i.e. none of the source files include first-level headings.

01-First-Steps-In-R contains an additional h1 heading in the page content. The heading levels in this page should be adjusted down.

• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.

• All exercises have solutions.

• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)

There are no exercises in this lesson. Can you add some? If you don't think they are necessary, please describe the other types of formative assessment that would be used when teaching the lesson in a workshop?

• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.

See my note about the copyright statements in the license file in the response for the Workshop Overview lesson.

• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.

• a list of lesson maintainers.

Please add a list of the maintainers of the lesson to the repository README.md.
The README for this lesson is a copy of the equivalent in the Lesson Example repository, and needs to be updated.

• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.

• setup and installation instructions.

The Setup page still includes the setup instructions for the lesson template, copied over from the Lesson Example site that you based the lesson on. These setup instructions should be replaced with content relevant to the lesson.

• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

Please add a glossary of key terms that appear in the lesson.

General

• Please adjust the contact email address (email in the _config.yml file) to point to someone who people can contact about the curriculum.

[tobyhodges]

Editor checks - Data processing and visualization for metagenomics

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

The images throughout the lesson have alternative text but in most cases it is very brief and undescriptive. Please expand the alt-text for these figures, so that they are more accessible to screen readers.

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.
  • no “jumps” are present between heading levels e.g. h2->h4.

h6 headings are used for image captions. If you want to use captions,
please replace these elements with something else, e.g. italicised text.

• no page contains more than one h1 element i.e. none of the source files include first-level headings.

Episodes include h1 headings in the page content. The heading levels in these episodes should be adjusted to convert these to h2.

• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.
• All exercises have solutions

In 08-hands_on-diversity, "Exercise 1 : Exploring geoms" is missing a solution.

• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)

• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

See my note about CC0 vs CC-BY in the response for the Workshop Overview lesson.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.

See my note about the copyright statements in the license file in the response for the Workshop Overview lesson.

• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.

• a list of lesson maintainers.

Please add a list of the maintainers of the lesson to the repository README.md.
The README for this lesson is a copy of the equivalent in the Lesson Example repository, and needs to be updated.

• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.

• setup and installation instructions.

Setup instructions are present but the page needs further modification, to remove instructions specific to the Data Carpentry Genomics lesson from which they are derived.
I recommend that you link from all lesson Setup pages to the Setup page of the Metagenomics Workshop Overview site, so that you only need to maintain a single version of the setup instructions for your whole curriculum.

• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

Please add a glossary of key terms that appear in the lesson.

General

• Please adjust the contact email address (email in the _config.yml file) to point to someone who people can contact about the curriculum.

[tobyhodges]

Following up with some more information that I hope will be helpful as you begin to work through the points raised above.

1. There is a lot of information in those responses, and I acknowledge that it may be difficult to keep track. We recommend that you open issues on the lesson repositories for each point raised, to help you maintain a view of what has been addressed as you go along.
2. I used the WAVE browser plugin from WebAIM when performing the accessibility checks described above. I can definitely recommend it as an easier way of checking things like image alt-text and heading levels.
3. You do not need to provide a written response to all of the points raised, but please post back here when you think they have been addressed. I can then run through the checklist again and confirm that the curriculum is ready to move to review. However, if you would like to provide additional explanation for any of the points raised (e.g. the license of the example data, or the lack of exercises in the Introduction to R lesson), I encourage you to do so.

[nselem]

Hello Toby,
We have been working on your corrections and we have some doubts.

1. Data were taken from the work of Okie et al 2020 that has a license Creative Commons Attribution License, I have written the authors and they wish to stay on that license. Also, I don't think they can change it. We understand that CC0 would be ideal but I feel it won't be possible.
2. To update the license of the lesson we should remove any reference to "The Carpentries"
   for example
   -"All Software Carpentry and Data Carpentry instructional material is made available under the Creative Commons Attribution license. The following is a human-readable summary of (and not a substitute for) the full legal text of the CC BY 4.0 license."

• "Software Carpentry" and "Data Carpentry" and their respective logos are registered trademarks of Community Initiatives.
  or just change the attribution section?

[tobyhodges]

Thank you for raising this, @nselem.

Data were taken from the work of Okie et al 2020 that has a license Creative Commons Attribution License, I have written the authors and they wish to stay on that license. Also, I don't think they can change it. We understand that CC0 would be ideal but I feel it won't be possible.

To be clear, I was suggesting only that the authors adjust the license to release the data into the public domain - CC-BY is an appropriate (and excellent) license for the article itself. I do understand that, as authors of the lesson, you are limited in how much control you have over this. We will discuss internally how to proceed here and I will come back with a suggestion soon.

To update the license of the lesson we should remove any reference to "The Carpentries"
for example

• "All Software Carpentry and Data Carpentry instructional material is made available under the Creative Commons Attribution license. The following is a human-readable summary of (and not a substitute for) the full legal text of the CC BY 4.0 license."
• "Software Carpentry" and "Data Carpentry" and their respective logos are registered trademarks of Community Initiatives.

or just change the attribution section?

I believe the attribution sections are the only parts that need changing. This is to ensure that you as authors receive the credit you deserve for writing the lesson, rather than The Carpentries.

[Czirion]

Hello Toby,

We have finished correcting the issues that you proposed, except that there are 4 pull requests awaiting to be merged because Nelly is not currently available and she is the only one with access to the Introduction to R for Metagenomics repository.

We remain attentive to any other corrections or indications.

Thank you so much for your review and your comments.

All the best,

Claudia

[nselem]

hello
I have merged the pull requests!

[tobyhodges]

Thank you @nselem @Czirion and others who have worked to address the points raised in the editorial checks. I am going to run through the checklist again for each of the lessons and will respond here again after that.

For now, I want to address the data set question: when the lessons have satisfied the other editorial checks, I will be happy to begin looking for reviewers for the lessons. Although I would prefer to see lessons use data in the public domain (and the requirements of the CC-BY license are largely meaningless for data), I must recognise that:

1. the data you have chosen otherwise appears to be a good fit to the lesson, and
2. it would be best for everyone if discussions of what data is most appropriate to use would take place earlier in the lesson development process.

We will work to improve the documentation, resources, and training we provide around this. That should put us in a better position to expect lesson authors to be able to comply with the requirement for CC0 data before they arrive at The Carpentries Lab.

That being said, exploring the Zenodo entry linked from https://carpentries-incubator.github.io/metagenomics-workshop/setup.html, I noticed that you are using the DOI specific to version 2 of the dataset (https://doi.org/10.5281/zenodo.6251246). Although this is currently the most recent version of the data, you would need to update the Setup page if you ever need to create a new version. If you change to using the persistent DOI (https://doi.org/10.5281/zenodo.4285900) in the Setup page, it will always point to the most-recent version, making your curriculum easier to maintain in the long run.

I will try to post again soon with updated editorial checklists for each lesson in the curriculum.

[nselem]

I didn't know there was a persistent zenodo DOI. Thanks for letting us know, I have already updated all references to it.
Nelly

[tobyhodges]

Editor checks - Metagenomics Workshop Overview

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.
• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.
  • no “jumps” are present between heading levels e.g. h2->h4.
  • no page contains more than one h1 element i.e. none of the source files include first-level headings.
• The contrast ratio of text in all figures is at least 4.5:1.

Some of the contrast is low in the AWS screenshots, but on balance I think it is better to include these screenshots than to omit them, and we really have no power to improve the contrast and accessibility of the AWS interface, so I am satisfied with this aspect of the lesson site.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.
• All exercises have solutions. not applicable
• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.) not applicable
• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

See my previous response about the data set, above.

Design

• Learning objectives are defined for the lesson and every episode. not applicable
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.
• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.
• a list of lesson maintainers.
• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises. not applicable
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson. not applicable

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.
• setup and installation instructions.
• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

The glossary is still missing but as I mentioned before, may not be required for an overview lesson site like this.

[tobyhodges]

Editor checks - Project organization and management

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

alternative text fields have been added for the images in the 03-ncbi-sra episode, but these fields are currently empty. Please add some text into these fields. I can recommend the resources I linked to in my previous editorial comment on this lesson to help you get started with it.

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.
  • no “jumps” are present between heading levels e.g. h2->h4.
  • no page contains more than one h1 element i.e. none of the source files include first-level headings.
• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.

• All exercises have solutions.

• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)

• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

See my earlier note about the data set.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.
• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.
• a list of lesson maintainers.
• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.
• setup and installation instructions.
• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

[tobyhodges]

Editor checks - Introduction to the Command Line for Metagenomics

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

Thank you for adding alternative text to three of the images. There is one more, which should be present in Exercise 2 of 02-the-filesystem where the image is currently not being displayed and no alt text is available. Please fix that image and add alt text for it before the lesson can be reviewed.

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.
  • no “jumps” are present between heading levels e.g. h2->h4.
  • no page contains more than one h1 element i.e. none of the source files include first-level headings.
• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.

• All exercises have solutions.

• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)

• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

See my earlier note about the data set.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.
• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.
• a list of lesson maintainers.
• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.
• setup and installation instructions.
• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

[tobyhodges]

Editor checks - Introduction to R for Metagenomics

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

Alternative text is still missing for the images in this lesson. For advice on how to write alt-text for these images, see the resources I linked in previous comments.

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.

The highest heading level in most episodes is still h3. The heading levels in these episodes should be adjusted so that sections begin with h2 headings. Within the Markdown content of an episode, the heading level is set based on the number of # symbols at the beginning of the line i.e. ### heading = <h3>heading</h3>, ## heading = <h2>heading</h2>.

• no “jumps” are present between heading levels e.g. h2->h4.
• no page contains more than one h1 element i.e. none of the source files include first-level headings.
• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.
• All exercises have solutions.
• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)

The table in Exercise 2 (in 03-dataframes) is not being displayed correctly.

• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

See my earlier note about the data set.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.
• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.
• a list of lesson maintainers.
• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.
• setup and installation instructions.
• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

[tobyhodges]

Editor checks - Data processing and visualization for metagenomics

Accessibility

• All figures are also described in image alternative text or elsewhere in the lesson body.

A few images in the lesson are still lacking alternative text. I found this resource very helpful when confronted with the challenge of writing alt-text for data visualisations: https://medium.com/nightingale/writing-alt-text-for-data-visualization-2a218ef43f81

• The lesson uses appropriate heading levels:
  • h2 is used for sections within a page.
  • no “jumps” are present between heading levels e.g. h2->h4.
  • no page contains more than one h1 element i.e. none of the source files include first-level headings.
• The contrast ratio of text in all figures is at least 4.5:1.

Content

• The lesson teaches data and/or computational skills that could promote efficient, open, and reproducible research.

• All exercises have solutions

• Opportunities for formative assessments are included and distributed throughout the lesson sufficiently to track learner progress. (We aim for at least one formative assessment every 10-15 minutes.)

• Any data sets used in the lesson are published under a permissive open license i.e. CC0 or equivalent.

See my earlier note about the data set.

Design

• Learning objectives are defined for the lesson and every episode.
• The target audience of the lesson is identified specifically and in sufficient detail.

Repository

The lesson repository includes:

• a CC-BY or CC0 license.
• a CODE_OF_CONDUCT.md file that links to The Carpentries Code of Conduct.
• a list of lesson maintainers.
• tabs to display Issues and Pull Requests for the project.

Structure

• Estimated times are included in every episode for teaching and completing exercises.
• Episodes lengths are appropriate for the management of cognitive load throughout the lesson.

Supporting information

The lesson includes:

• a list of required prior skills and/or knowledge.
• setup and installation instructions.
• a glossary of key terms or links out to definitions in an external glossary e.g. Glosario.

[tobyhodges]

Thank you again for the time you have taken to work through the comments I left from the first round of editorial checks: it was a genuine pleasure to work through the lesson sites again and see that so many of the points have been dealt with quickly and effectively.

I have posted new editorial checks above, with the majority of remaining issues relating to alternative text for images. If you can spend a bit more time getting those alt-text fields written, these lessons will be ready for review.

As always, if you have any questions and/or would like more information about anything I have written above, please reply in this thread and I will be happy to discuss further and ensure that you are able to keep making progress.

[Czirion]

Hello Toby
The remaining issues have been solved. Thank you so much for your comments.

[tobyhodges]

Thank you @Czirion and everyone for your work to address my comments from the editorial checks. I'm pleased to be able to pass this over to reviewers.

@nselem @Czirion Please add the code below to the README.md of each of your lesson repositories. This will add a badge to indicate that the lessons are undergoing peer review, and display the current status of the review. It will automatically update as the lessons move through the steps of the review process.

[![The Carpentries Lab Review Status](http://badges.carpentries-lab.org/11_status.svg)](https://github.com/carpentries-lab/reviews/issues/11)

[tobyhodges]

@fpsom & @PaulaNietoG thank you for volunteering to review lessons for The Carpentries Lab. Please can you confirm if you are happy to review this Metagenomics curriculum?

You can read more about the lesson review process in our Reviewer Guide.

[tobyhodges]

Thank you so much for these reviews @PaulaNietoG 🙌 If you have any suggestions for the Workshop Overview site, that were not already covered by @fpsom's comments above, please feel free to post those here as well.

@AbrahamAvelar, @aaronejaime, @bwanya, @fabel134, @Vanessaarfer, @Czirion, @Bedxxe, @nselem, @EdderDaniel, and @ahmedmoustafa: Reviews are now complete, and you can proceed with making changes and responding to the comments and suggestions whenever you are ready.

@PaulaNietoG & @fpsom thank you again for volunteering your time to review this curriculum. Please stay subscribed to this thread, so that you can check changes made in response to your review, and in case the lesson developers need to discuss any of your comments while they incorporate the feedback.

[nselem]

Dear @tobyhodges @fpsom and @PaulaNietoG
We have considered all your valuable suggestions and worked on them as individual issues in the Metagenomics lesson. We thank you very much for your detailed observations that have helped us to improve the material significantly.
A summary of the main changes is the following list:

• The setup page was cleaned and improved. Now it explains how to use the different options for obtaining the software; the Amazon Web Services remote machine and the local installation of the programs using a Conda environment in the three main operating systems. For AWS, a new instance version has been created, and instructions have been updated. All the software is now included in the conda environment for the local installation.
• A copy of all the files needed while running the lesson was added as a hidden directory and uploaded in Zenodo. This backup can be used if a learner cannot generate some intermediate files. Adapter files are included in the "untrimmed" folder where it is needed. The Amazon Image was updated with this new data folder and the Conda environment. This change is explained in the Instructor Notes.
• To diminish the cognitive load in the Data Processing and Visualization for Metagenomics, the content was rearranged and split into four instead of three episodes r-based episodes. Now, in the first one the OTU table is explained, the other talks about diversity, and finally, in the last one abundance graphics are produced. Some of the content was moved to the Introduction to R lesson, so the taxonomy episodes are a little less dense.
• To diminish the cognitive load in the Introduction to the Command Line, the scp command was left as optional. We use the export function in the r studio interface as an alternative. In our experience in the workshop, this was a difficult part for learners, nevertheless, it is valuable, so we left it there so they can come back and keep learning if they want. The rest of the material, in our experience, works well even for newcomers in the given amount of time.
• The exercises and discussions were improved, and solutions were added to the exercises that did not have them. New conceptual discussions were added to several episodes.
• Figures were complemented with missing alternative text.
• A general review of the typos and writing was made in all the episodes.
  Now I will point to the issues that address punctual observations that need to be improved.
• New and cleaned setup: issue#62
• Typos issue#61
• Adding conceptual questions issue#60
• Provide all required filesissue#59
• Adapter files and multiline command in Trimming and Filtering episode issue#58
• Exercise 2 in Assessing Read Quality episode. issue#57
• Cognitive load in Data processing and visualization for metagenomics and splitting into smaller episodes. issue#56
• R datatypes issue#54
• Some Spanish names. issue#53
• Misleading name for R lesson issue#52
• Improve color palette for color blindness. issue#51
• Repeated Learning Objectives issue#50
• Intro to the command line. issue49 and simplified version of Writing Scripts and Working with Data
• Improve Writing Scripts and Working with Data episode. issue#48
• Improve Redirection episode. issue#47
• Working with Files and Directories episode issue#46
• Introducing the Shell episode issue#45
• Alternative text of images #44
• Improve discussions in Project Organization and Management lesson issue#43
• Image transcription in Examining Data on the NCBI SRA Database issue#42
• Update the zenodo link in Examining Data on the NCBI SRA Database issue#41
• Improve Structuring data in spreadsheets in Data Tidiness issue#40
• Improve Data about the experiment in Data Tidiness issue#39
• Add confidentiality in Data Tidiness issue#38
• Improve Examining Data on the NCBI SRA Database episode #37
• Workshop prerequisites issue#36
• Develop Instructor Notes issue#32

[tobyhodges]

Thank you @nselem et al for taking time to address Reviewers' comments, and for providing the itemised list above.

@fpsom and @PaulaNietoG: please take some time to look through the updated versions of the lessonsand post back here to let the authors know if there are any additional changes you would like them to make, or if you recommend that we accept the curriculum to The Carpentries Lab. You may find it helpful to refer to the Reviewer checklist again if you would like to use that as a guide.

[tobyhodges]

Pinging @fpsom and @PaulaNietoG again: have you had time to look through the curriculum, after the authors completed their changes in response to your initial reviews?

[PaulaNietoG]

Hello! Everything looks good to me. I recommend that the lesson is accepted. Congratulations on the hard work!

[fpsom]

Hi everyone! Really sorry for the long delay in replying. I will echo @PaulaNietoG; the lesson looks great and I am quite eager to try teaching it :) And many thanks for all your hard work!

[tobyhodges]

Thank you @PaulaNietoG and @fpsom, this is very exciting. I know it takes considerable time to conduct a review on a whole curriculum like this, and I really appreciate the thorough and thoughtful approach you have both taken to it. I will follow up with you separately to request some feedback on your experience as reviewers.

@nselem @AbrahamAvelar, @aaronejaime, @fabel134, @Vanessaarfer, @Czirion, @Bedxxe, @nselem, @EdderDaniel, @bwanya, and @ahmedmoustafa I am delighted to be able to mark the curriculum as approved now. There are a few other steps to take, to transfer the curriculum into The Carpentries Lab and pass on the submission to JOSE. I am traveling today so may not get through everything before the weekend, in which case I will come back to the task at the start of next week.

[nselem]

Dear @tobyhodges, @PaulaNietoG and @fpsom, thank you for your time and effort that helped us improve the lesson. It has been an enriching experience for the team. @fpsom We hope that soon you can teach the lesson, we will be very happy.

[tobyhodges]

The lesson repositories have now all all been transferred into The Carpentries Lab. Here are the new repository locations:

• https://github.com/carpentries-lab/metagenomics-analysis/
• https://github.com/carpentries-lab/metagenomics-R/
• https://github.com/carpentries-lab/metagenomics-organization/
• https://github.com/carpentries-lab/metagenomics-workshop/
• https://github.com/carpentries-lab/metagenomics-shell/

Note that some of the repository names have changed, as well as their GitHub organisation. Those of you with local clones of these repositories should update your remotes accordingly (use git remote set-url for this).

I also renamed the default branch of metagenomics-organization to main, and anyone working with a local clone of that repository should run the following commands while working on their local master branch:

git branch rename master main
git push -u origin main

Finally, I recommend that you now publish a new release of each lesson repository to Zenodo.

[nselem]

By publishing the lesson, do you mean uploading all the local repositories to Zenodo? or just the data? If the answer is the repositories, should be one upload for all the five repositories, or separate uploads for each one?

[Czirion]

I was checking the new websites and it is so exciting to see "This lesson has passed peer-review!" in the lessons!
Thank you so much.
I just realized that this link https://carpentries-lab.github.io/metagenomics-organization/ for the Organization lesson is not working, and I couldn't find an error.

[tobyhodges]

Good catch, @Czirion, thanks. That was an unanticipated side effect of renaming the default branch - GitHub Pages was deactivated. It should be back up now.

[tobyhodges]

By publishing the lesson, do you mean uploading all the local repositories to Zenodo? or just the data? If the answer is the repositories, should be one upload for all the five repositories, or separate uploads for each one?

At the moment, I think only the data for the curriculum is published on Zenodo. So far, I have set that DOI to be linked from the banner at the top if each lesson site. However, I recommend that you now create a Zenodo entry for each lesson as well, which you can populate with a list of authors, maintainers, reviewers, etc. When those have been made, the _config.yml file can be updated in each repository with the relevant DOI information. This will make it easier for people to cite your lessons, and in future you can update the Zenodo entry with new releases of each lesson when you think those are necessary.

If and when the curriculum is accepted to JOSE, we can update the DOIs in the _config.yml files again, to point to the publication.

[tobyhodges]

Since the curriculum was accepted to JOSE yesterday, I am going to close this issue.

Congratulations again to all the reviewers, and thanks one more time to @fpsom and @PaulaNietoG for reviewing 🙌

[fpsom]

Well done, congratulations! :)