tops_faq.md

Transform to Open Science (TOPS) Frequently Asked Questions

This page is a living document of the most common questions posed to the TOPS team. It is our hope that these answers will benefit the wider community. Further details can be found in the transcripts of our community events on our community forum and community panel pages.

General Questions about Open Science

What is open-source science (OSS)?

Open-source science is a commitment to the open sharing of software, data, and knowledge (algorithms, papers, documents, ancillary information) from the start of research activities. The principles of OSS are to make publicly funded scientific research transparent, inclusive, accessible, and reproducible. OSS is enabled by advances in technology, including collaboration tools and cloud computing. More information is available from NASA's Science Mission Directorate (SMD) Policy Document (SPD-41) on science information policy.

What is the difference between open-source science and open science?

Ramachandran et al. define open science as “a collaborative culture enabled by technology that empowers the open sharing of data, information, and knowledge within the scientific community and the wider public to accelerate scientific research and understanding.” The primary difference is that open-source science commits to making the scientific process open from the start of research activities rather than making research results open once the research is complete and papers are published. The commitment to conduct research in the open supports greater participation in answering fundamental scientific questions and the use of publicly funded research, data, and analysis for societal benefit.

What is the difference between open-source science and open data?

Open data are a critical component of open-source science. Other components of OSS include open documentation, publications, citizen science, challenges/prizes, open-source software, open peer review, open notebooks, and open educational resources among others.

Is the lack of open science a cultural or technical issue?

Both. Open science is more than just the open sharing of data and code. It also is a cultural shift in the scientific process that encourages collaboration among people of diverse backgrounds, including scientific field, gender, location, ethnicity, and expertise. By removing barriers to participation in the scientific process, open-source science is inherently inclusive and collaborative. NASA’s vision is to use open science principles to expand participation in the scientific process, improve reproducibility, and accelerate scientific discovery for societal benefit. Technological considerations include use of existing investments in infrastructure and mechanisms for community contributions, while limiting the proliferation of unvalidated data.

Does open science mean "free" science?

Open science is the commitment to the full, free, and open sharing of data, code and knowledge as early in the research process as possible. In terms of activities related to NASA’s Open-Source Science Initiative, Science Mission Directorate (SMD) Policy Document (SPD-41) consolidates existing guidance for the openness and accessibility of data, software, papers, and ancillary information resulting from SMD-funded efforts

What is the first step to getting involved with open science?

The first step is to support open-source science efforts within your communities. The Turing Way is a community-driven guide that provides more details on how to design open projects. TOPS is developing an open science curriculum that will become available in late 2022. The UNESCO recommendations have detailed definitions and suggestions on areas of action to support open science.

What incentives and disincentives are being used to encourage open science?

NASA awards funding based on the strength of the scientific ideas and the ability to advance those for societal benefit. Traditionally, the enterprise has awarded individuals, but without a focus on openness; incentives have been based on publishing papers in big-release journals. However, NASA will take the next year to shift incentive structures from what they have been in the past, to include incentivizing open science activities (eg. collaborations, team-building, open data, open software, and open-access publications). This shift requires NASA to engage with professional organizations, academia, etc. to make it happen and be appropriately recognized. It is important to note that this shift is not automatic but rather will require some experimentation to see what works and what does not. It is also imperative that NASA get feedback from the community to ensure the effectiveness and efficiency of its OSS efforts.

How do I know that my code will not be taken without proper attribution?

One of the principles of open science is proper attribution of previous work, collaborations, and knowledge used from various sources. Making your code open, appropriately licenced, and assigning it a DOI will help researchers track code being developed. The open development of code should make it easier to identify when people aren’t properly attributing their work.

Can you recommend some resources to help people develop their research using open science principles?

One of the goals of TOPS is to develop resources to help researchers, organizations, and citizen scientists do their work using open science practices and principles. TOPS is developing an open science curriculum that will become available in late 2022. Please sign up for our newsletter to receive the latest updates!.

How can I advocate for open science?

A few ways to get started are: Make data non-proprietary and available in an open repository; Provide datasets in standardized formats and assign them DOIs; Develop open-source software and code, using best practices and rigorous version control, so that people can reuse it; Support community development and encourage reuse; Publish in open-access journals; Actively engage the public through storytelling (blogging, social media), hack-a-thons, and citizen science; and cite your data, software, and documentation.

Can you expand a bit more on how NASA is thinking about open science in terms of the research outputs domain?

TOPS is advocating a vision of open science in which the entire research workflow--from inception to the creation of data and software artifacts and publishing results--is as open as possible.

What kind of training and educational initiatives could we implement to make open science more accessible?

OpenCore is just the beginning; TOPS will announce in late 2022, the upcoming Year of Open Science which will include a comprehensive plan on engaging with the scientific community through hackathons and summer/winter schools, and at all the large science society conferences. Check out our calendar to join us!

How can research and data-driven artists further science?

One way to broaden participation is through making science more accessible and the arts are one way to do that. We hope to have a Space Apps challenge focused on STEAM initiatives to get people from all of the world thinking about this.

How does citizen science connect to open science?

We realize that the road to making open science a reality doesn’t begin and end with academics and NASA scientists. We want to reach science-interested populations too! Citizen science provides an opportunity for the general science-interested public to get involved with scientific research to address societal needs, particularly those at a regional or local level, and to advance innovation.

How can data scientists and machine learning experts help further open science?

There are open science principles that those working with code and data can incorporate into their work, even if it is not “traditional” scientific research. They can make the underlying data findable, accessible, interoperable and reusable (which is known as the FAIR principles). Any code which is developed should be as open as possible (e.g., open-source or white-listing); including the creation of clear documentation so that others can build on your work.

General Questions about TOPS

NASA TOPS is a $40M mission over 5 years. How will those funds be spent?

TOPS is across all of NASA’s Science Mission Directorate (SMD). The majority of the funding is to support training activities to help the SMD community learn about Open Sciences. This includes training that is tailored to the needs of the scientific communities of SMD. Opportunities will be announced that are available to all directorates and the different directorates are participating in the design and development of the project.

What are the concrete objectives with the TOPS mission? What outcomes and deliverables are you pursuing?

TOPS goals are to enable

1. 20,000 individuals to earn an Open Science Badge,
2. lead to at least five major discoveries, and
3. increase participation of underrepresented groups by two-fold.

For goal 1, we will know the exact amount of people completing the course thanks to registration. For goal 2, we plan to solicit proposals in 2023 for high risk, high reward, science projects, across all of NASA science (e.g., planetary, heliophysics, astrophysics, biological and physical sciences, and Earth science), that are open from inception. These projects will be required to follow open science best practices. We will be able to measure success by whether they achieve the goals stated in their proposals, and contribute to a major advance in their field. For goal 3, we have internal metrics that we hope to make public, and can track participation by under-represented communities on NASA science teams, funded proposals, committee, and review panels. While these specific metrics may allow us to see if we reach our stated goals, we need to be able to measure our progress against our objectives as well: increasing adoption of open science and increasing participation in science by historically excluded groups. That requires a more nuanced set of metrics that we are continuing to develop.

What is the plan for collaborating across groups?

Open-source science (OSS) is a part of open science. Open-source science is a commitment to the open sharing of software, data, and knowledge (algorithms, papers, documents, ancillary information) from the start of research activities. The principles of OSS are to make publicly funded scientific research transparent, inclusive, accessible, and reproducible. All of these aspects of open-source science are intentionally a part of open science.

What does public engagement with open science look like for people who don't want to be scientists or coders?

We realize that the road to making open science a reality doesn’t begin and end with academics and NASA scientists. We want to reach science-interested populations too! Citizen science provides an opportunity for the general science-interested public to get involved with scientific research to address societal needs, particularly those at a regional or local level, and to advance innovation.

Are engagement plans aimed within the US or internationally?

TOPS is engaging both nationally and internationally. The focus for 2023 is national in scope; yet, we continue to engage and collaborate with other international space agencies and organizations to ensure the open science reach is more broadly distributed.

Have actionable plans been made to ensure that communities historically excluded from research are reached by TOPS?

Yes; plans have been developed that include engagement and support for these communities. A core value within the TOPS team is to meet historically excluded groups where they are, and we will continue to follow through with this value for the next 5 years. We are working to have a strong presence at conferences that focus on historically excluded groups. We have partnered with NASA’s Minority University Research and Education Project (MUREP) to support different activities such as funding 3-year NASA internships and to develop future solicitations. Our TOPS Community Panel includes leaders in open science and is 50% women and 70% people of color. We will also be developing further plans based on the engagement and feedback we receive. We welcome suggestions, contributions, and comments through our GitHub site or by contacting us directly.

Is there a forum or mechanism to share experiences on open science with this community?

The TOPS GitHub is a great source of information, and is also a great way to share experiences and ask questions. The discussion can be found at https://github.com/nasa/Transform-to-Open-Science/discussions .

Where can I go to get more information on NASA open data policy?

Please refer to the official website at https://science.nasa.gov/researchers/science-data/science-information-policy. .

How does the OSTP Public Access Memorandum affect NASA's policies and activities related to Open Science?

The 2022 memorandum titled, Ensuring Free, Immediate, and Equitable Access to Federally Funded Research, will take critical steps to achieve equitable delivery of Federally funded research results and data to all of America. This is a very exciting step in creating more equitable access to the incredible research NASA funds. Additionally, NASA plans to release an updated Scientific Information Policy, SPD-41a this Fall. This policy describes how scientific information (publications, data, and software produced as part of scientific research activities) produced from SMD funding is shared. The draft was released 11/2021: https://science.nasa.gov/researchers/science-data/science-information-policy. .

How will you enable open meetings? Will they all be virtual? Will funding be provided for volunteers to present at science meetings?

We are enabling open meetings by providing an option for virtual attendance, including the ability to submit questions. We have plans to expand participation at NASA science team meetings; please subscribe to our listserv to hear more.

Is there a specific place in the Github repository(ies) that you shared that lists these meetings/conferences/societies?

Please refer to the TOPS priority events for 2023 list on our GitHub.

How do I access the recordings of the community forum meetings?

The recordings are available on our GitHub Community Forum page; in addition we are posting all of the forums on the Science@NASA YouTube site, under the TOPS playlist.

If I declare a year of open research across disciplines at my own institution, how do you encourage us to engage with TOPS?

Declaring a year of open research at a particular institution will depend on the type of institution. We encourage everyone to engage with TOPS as much as able and we are providing resources through our GitHub site to be able to do so.

Are you coordinating this mission with the other federal US agencies that provide research funding (e.g., NSF, NOAA, DOE)?

Formal and informal partnerships with other agencies and organizations will be announced soon!

Where can we get TOPS stickers and pins?

You can visit us at conferences to find our latest educational and informational resources. We are also working on a distribution mechanism to share resources with the full community, so join our listserv to stay tuned!

Incentives and Recognition for Open Science

What are NASA's plans to make sure that the extra effort required to make work open is properly recognized?

There are different ways to recognize effort. First, there is financial recognition. If research is open from inception, and openness has been planned and budgeted for at the proposal stage, then being open isn’t a substantial extra effort. So, with regards to cost, the effort will be recognized by funding it in the budget. Then there is professional recognition. If one put in the effort to be open, one should certainly be recognized. There is already a lot of evidence that by being open, science has more impact and more citations. But we are planning additional specific activities to incentivize and recognize open science activities. Further, TOPS is working with partners to create awards specifically created to recognize scientific achievements in open science.

How will researchers who are used to working in more closed frameworks be incentivized to share their work products?

Many researchers adopt open science principles as they become familiar with them and as they become best practices in their community. TOPS first goal is to increase the understanding of open science practices to support the culture shift. SMD is also supporting incentives to adopt open science through solicitations such as the F.8 Supplement for Open Source Software and also developing initiatives that support open science so that it is easy to share the scientific information that is produced. It is important to note that this shift is not automatic; this process will require some experimentation to see what works and what does not. It is also imperative that TOPS get feedback from the community to ensure the effectiveness and efficiency of its efforts.

Can we consider and incorporate published literature in the form of books, book chapters, and papers related to open science?

Absolutely! We believe that to be successful in spreading the word on open science we need to be as visible as possible, including by publishing articles, papers, and blog posts about our successes and failures.

Can the public participate in solicitations?

NASA solicits research proposals through Research Opportunities in Space and Earth Sciences (ROSES). More information on ROSES is available on https://science.nasa.gov/researchers/sara/grant-solicitations.

Getting Involved in TOPS

Who can participate?

Anyone with an interest in open science is welcome to participate in TOPS! Sign up for the TOPS newsletter to learn how to get involved and keep up-to-date with the latest TOPS activities.

How can you become part of the Community Advisory Panel for TOPS?

The Community Panel for 2022 has been selected. However, each year we will have an open call for new members. Please continue to monitor the GitHub as all announcements will be provided there.

How can students at different levels get involved?

Anyone with an interest in open science is welcome to participate in TOPS! Sign up for the TOPS newsletter to keep up-to-date with the latest TOPS activities, including student internships. Contribute to the discussion on our GitHub. Further, the open science curriculum that TOPS is developing will be free and open for all regardless of disciplinary or education level.

What specific plans do you have for outreach that will reach different communities?

See our calendar for a list of conferences we will be attending this year and next.

Is TOPS engaging with open science users across scientific domains?

Yes! Subject matter experts for building the OpenCore stretch across scientific fields.

How will TOPS ensure that everyone, working in an open-science environment, understands technologies through the same lens?

TOPS will be releasing competitive NASA funding opportunities for add-ons to this curriculum, such as discipline-specific modules or using data in the cloud. How to effectively use different compute environments could be part of such an extension.

The TOPS Open Science Curriculum

What is the plan for the translation of the open science curriculum into other languages?

TOPS will be releasing competitive NASA funding opportunities for add-ons to this curriculum, such as discipline-specific modules or using data in the cloud. How to effectively use different compute environments could be part of such an extension.

How are you incorporating existing educational resources (e.g. Mozilla, The Turing Way) into these modules?

Content subject matter experts (SMEs) are bringing existing material with them to draw from and reuse. We are using existing examples, but there is an opportunity to connect these efforts while drawing on already existing material. Where existing material exists, with licensing that allows reuse, we are happy to include or reference with appropriate attribution.

Are there any plans to introduce high school students or college students to open science?

The initial version of OpenCore will be tailored to researchers and scientists interested in further exploring open science and receiving the NASA TOPS Open Science certification. OpenCore will be free and open for all regardless of disciplinary or education level.

When will the curriculum be available and will the modules be accessible online for people to complete at their own pace?

OpenCore curricula will be available through the Open EdX platform in April 2023. We will be having the official Year of Open Science launch at the American Meteorological Society and American Astronomical Society January 2023 meetings.

Do we have specific subject or scientific domains prioritized?

TOPS is a NASA Science Mission Directorate initiative focused on ensuring an equal playing field for all scientists and researchers regardless of domain, tenure, or affiliation.

The planned modules emphasize data and software; is there an intention to also discuss different types of compute environments?

The hard part in designing the modules is deciding what NOT to include. Modules have to be specific enough to be useful, but general enough to appeal to those who are new to any scientific field. They have to be short enough that people make it to the end, yet engaging and easily digestible. OpenCore is just the beginning and foundation of how to do open science, please stay connected as NASA TOPS will continue to expand upon the foundational courses in the near future.

Will these training targets NASA-specific considerations for using open science tools?

The open science modules created as part of the OpenCore aim to be as inclusive of toolsets, resources, and research environments as possible.

What is the curriculum advising as the mechanism for when open science collides with AI ethics (e.g. identification of informal settlements)?

For OpenCore we discuss responsible research practices as being fully integrated into Open Science practices. We indicate that learners should seek specific ethical behavioral expectations from their institutions, agencies, societies and other types of researcher affiliations to navigate these potential conflicts.

Is TOPS focused on supporting open source scientific computing or on open source computing that empowers science?

TOPS will be releasing competitive NASA funding opportunities for add-ons to this curriculum such as discipline-specific modules or using data in the cloud. How to effectively use different compute environments could be part of such an extension.

There is a lot of concern about data information security; what is the curriculum advising with regards to how we make the products of our research unassailable and reproducible?

Data information security is a complex topic. We won’t be going in depth in the OpenCore modules on security per se since that depends on the physical environment available to a researcher. We will address the purpose for persistent identifiers, good metadata, licensing for reuse, and elements around reproducibility.

Will there be a pilot phase of the OpenCore courses?

Initial review of the OpenCore content will occur in mid-July. The initial testing of the full OpenCore will occur in October. Iterative updates will be guided by Learning Management System data analytics.

Will the courses be pre-recorded? Live training? Watch videos?

The course materials will be available online for those wishing to take the course virtually, but will also be hosted at conference workshops in-person.

How might social scientists perceive, make use of, and contribute to this curriculum?

NASA TOPS believes that to change the world, we need everyone. If you are interested in engaging with us, please reach out to our TOPS team or submit an issue or discussion on our Github!

OpenCore Contributions

Who will be the content creators for OpenCore?

The content creators are open science subject matter experts in the community who were selected via a competitive, open process that was widely advertised. NASA TOPS intends on continuing to build upon the curriculum on an annual basis in support of the 5 year mission.

How will AGU be selecting content experts and testers for the modules?

If you sign up and say you want to help, we'll find a role. We had a very difficult set of criteria for evaluating content experts. Our process for the selection of the module leads resulted in an incredibly diverse group. They're very passionately engaged and, while they will help guide the content coming together, they will by no means take full responsibility for that content. We need everyone to make sure that we didn't miss anything and to ensure that what we build makes sense to every discipline and in an international context. We do have a rigorous set of review criteria; for example, we really want the language we're using to be easily translatable, for the language we use to not be offensive to anyone. The management team will be responsible for keeping modules in sync, releasing version control, change management, all the things that come with that.

Can subject matter experts volunteer to contribute to the course?

Across all of the open science topics that we are hoping to highlight, there are experts with valuable experience and content. TOPS invites everyone to participate in development of the curricula as a reviewer or a tester. You can get more details by signing up for the TOPS email list on the main GitHub page.

Is there a plan or standards about engaging and crediting organizations who contribute to the modules significantly?

One of the principles of open science is proper attribution of previous work, collaborations, and knowledge used from various sources. The communnity development of OpenCore is a key TOPS value and we encourage all contributions.

The Year of Open Science

What is the Year of Open Science?

Transform to Open Science (TOPS) is a five-year initiative designed to support scientists, agencies, organizations, and communities as they transform to an inclusive culture of open science. Many TOPS activities will be focused around 2023 as NASA’s Year of Open Science. In partnership with major scientific organizations, the Year of Open Science is designed to spark change and inspire open science engagement through events and activities that will shift the current paradigm.