chore(docs): New "Creating a Source Plugin" tutorial (#37538)

Co-authored-by: Ty Hopp <tyhopp@users.noreply.github.com>
Co-authored-by: Tyler Barnes <tylerdbarnes@gmail.com>
Co-authored-by: Alex Reed <aghreed@gmail.com>
Co-authored-by: pieh <misiek.piechowiak@gmail.com>
Co-authored-by: Michal Piechowiak <misiek.piechowiak@gmail.com>
Co-authored-by: Stephanie <52582720+stephmarie17@users.noreply.github.com>
Co-authored-by: Sarah Etter <sarah@sarahetter.com>

docs/docs/reference/config-files/node-api-helpers.md

@@ -9,8 +9,7 @@ showTopLevelSignatures: true
 
 The first argument passed to each of [Gatsby’s Node APIs](/docs/reference/config-files/gatsby-node/) is an object containing a set of helpers. Helpers shared by all Gatsby’s Node APIs are documented in [Shared helpers](#apis) section.
 
-```javascript
-// in gatsby-node.js
+```js:title=gatsby-node.js
 exports.createPages = gatsbyNodeHelpers => {
   const { actions, reporter } = gatsbyNodeHelpers
   // use helpers
@@ -19,13 +18,14 @@ exports.createPages = gatsbyNodeHelpers => {
 
 Common convention is to destructure helpers right in argument list:
 
-```javascript
-// in gatsby-node.js
+```js:title=gatsby-node.js
 exports.createPages = ({ actions, reporter }) => {
   // use helpers
 }
 ```
 
+The [Creating a Source Plugin tutorial](/docs/tutorial/creating-a-source-plugin/part-4/) explains some of the [Shared helpers](#apis) in more detail.
+
 ## Note
 
 Some APIs provide additional helpers. For example `createPages` provides `graphql` function. Check documentation of specific APIs in [Gatsby Node APIs](/docs/reference/config-files/gatsby-node/) for details.

docs/docs/tutorial/creating-a-source-plugin/part-0/index.mdx

@@ -0,0 +1,116 @@
+---
+title: "Part 0: Introduction and Prerequisites"
+---
+
+import { LinkButton } from "gatsby-interface"
+import { MdArrowForward } from "react-icons/md"
+
+## Introduction
+
+Welcome to the "Creating a Gatsby source plugin" tutorial. We're excited you're here!
+
+In this tutorial you'll learn how to create a Gatsby source plugin that follows best practices, uses Gatsby's latest features, and aligns with how the Gatsby team wants you to build source plugins. By following our instructions, your plugin will have great developer experience for you, and great reliability and usability for your users.
+
+Source plugins are an integral part to Gatsby's [GraphQL data layer](/docs/reference/graphql-data-layer/) as they are responsible for bringing data into that layer. They play a key role in hiding complex pieces of software (e.g. fetching and normalizing data from a remote API) behind an easy-to-use plugin. This also means that they have to be reliable, performant, and use best practices behind the scenes.
+
+There is already a wealth of [existing source plugins](/plugins?=gatsby-source) in the Gatsby ecosystem you can use, but there may come a time when you want to create your own source plugin that doesn't already exist. If that sounds like a situation you've found yourself in, this tutorial is for you! Otherwise if you're curious to learn more about how Gatsby's source plugins work, you'll also find useful information here.
+
+<Announcement>
+
+This tutorial is intended to be an **additive guide**. Each part builds up on each other so we recommend you going through it from start to finish. However, if you want to revisit a specific topic each part should have all the necessary information on its own to be informative.
+
+You'll have a functioning source plugin at the end of part 2. The later parts are improving the plugin, adding functionalities, and explaining advanced topics. So while you can already publish what you have after part 2, we highly recommend going through all parts to learn how you can further improve your plugin.
+
+</Announcement>
+
+By the end of this part of the tutorial, you will:
+
+- Identify what background knowledge is assumed throughout the tutorial
+- Describe what a source plugin is and why you would want to create one
+- Install the necessary software
+
+## Background knowledge
+
+This tutorial assumes a basic understanding of Gatsby, [GraphQL](https://graphql.org/), [Node.js](https://nodejs.org/en/), and [TypeScript](https://www.typescriptlang.org/).
+
+<Announcement>
+
+Don't worry, you don't need to be an expert with these! A high-level understanding of the basics should be enough. You'll pick up a lot through the course of the tutorial.
+
+</Announcement>
+
+In terms of Gatsby-related knowledge, you should first complete the [Getting Started Tutorial](/docs/tutorial/getting-started/) since this guide builds on the knowledge established there.
+
+In terms of the other technologies, you should be comfortable writing GraphQL queries, importing/exporting JavaScript modules, and making use of TypeScript types.
+
+## What is a source plugin?
+
+A plugin that adds additional data sources (content and data backends, e.g. content management systems like Contentful, WordPress, Drupal) to Gatsby that can then be queried by your pages and components. You can access these created nodes via Gatsby's universal GraphQL data layer.
+
+At a high-level, a source plugin:
+
+- Sources data from an external API in the best possible way
+- Creates new nodes and establishes relationships between them
+- Customizes your site's GraphQL schema
+- Improves the functionality of your external API by e.g. enabling advanced image manipulations through [Image CDN](/docs/how-to/images-and-media/using-gatsby-plugin-image/#gatsby-cloud-image-cdn)
+
+In terms of code, a source plugin is the same as other plugins in that it must include:
+
+- A `package.json` file with an entrypoint
+- One or more Gatsby API files (usually a [`gatsby-node`](/docs/reference/config-files/gatsby-node/) file)
+
+The difference is that source plugins should care only about sourcing data, while other plugins such as [transformer plugins](/docs/how-to/plugins-and-themes/creating-a-transformer-plugin/) or styling plugins may have other functionality that don't involve sourcing. Separating these concerns makes each plugin more reusable.
+
+## Why create a source plugin?
+
+The most common reason why you might want to create a new source plugin is that it may not exist yet in Gatsby's [plugin library](/plugins/?=gatsby-source).
+
+Source plugins are a powerful concept. In a Gatsby site you can combine many data sources together via source plugins, like commerce data from Shopify, or content from one or more content management systems (like Contentful, WordPress, etc.), all in a single unified graph. Gatsby then leverages this graph to offer exciting features like [incremental builds](/blog/2020-04-22-announcing-incremental-builds/) and [Image CDN](/docs/how-to/images-and-media/using-gatsby-plugin-image/#gatsby-cloud-image-cdn).
+
+Both of these topics will be covered in later parts of this tutorial, so stay tuned!
+
+<Announcement>
+
+**Please note:** If your data is local, e.g. on your filesystem or part of your site's repo then you generally **don't** want to create a new source plugin. Instead, use [`gatsby-source-filesystem`](/plugins/gatsby-source-filesystem/) as it already handles reading and watching files for you. You then can combine it with a [transformer plugin](/docs/how-to/plugins-and-themes/creating-a-transformer-plugin/#what-do-transformer-plugins-do), e.g. [`gatsby-transformer-remark`](/plugins/gatsby-transformer-remark) to transform markdown files.
+
+</Announcement>
+
+## When to create a source plugin?
+
+Gatsby plugins are most often thought of as modules that are published to [npm](https://www.npmjs.com/) so that they can be shared and installed by others, but they can also be [local plugins](/docs/creating-a-local-plugin/) that exist only in a single site.
+
+This same logic applies to source plugins. If you create a source plugin that you think others might make use of and you care to maintain it, please do publish it and [submit to the Gatsby plugin library](/docs/how-to/plugins-and-themes/submit-to-plugin-library/)! However, there is certainly no obligation to publish your plugin. If it serves its purpose for your use case just fine, keep it a local plugin.
+
+The parts at the end of this tutorial will teach you how to go about developing a plugin in a way that makes it easy to publish later.
+
+## Development environment
+
+The development environment is identical to the [Getting Started tutorial](/docs/tutorial/getting-started/), with the addition of [yarn](https://yarnpkg.com/), a package manager like [npm](https://npmjs.com). Refer to the [installation guide](/docs/tutorial/getting-started/part-0/#installation-guide) for setup instructions, as well as the [yarn installation instructions](https://yarnpkg.com/getting-started/install).
+
+This tutorial uses [yarn workspaces](https://yarnpkg.com/features/workspaces) to develop the plugin and its companion example site in a monorepo. After going through this guide feel free to use any monorepo tooling you'd like (e.g. pnpm, NX, Turborepo, etc.) as it's not relevant for the Gatsby plugin itself.
+
+## Summary
+
+Now that you know the basics about Gatsby source plugins and have your computer set up, you're ready for the tutorial!
+
+<Announcement>
+
+**Share Your Feedback!**
+
+Our goal is for this tutorial to be helpful and easy to follow. We'd love to hear your feedback about what you liked or didn't like about this part of the tutorial.
+
+Use the "Was this doc helpful to you?" form at the bottom of this page to let us know what worked well and what we can improve.
+
+</Announcement>
+
+### What's coming next?
+
+In Part 1 of the tutorial, you'll set up your project and learn how the project is structured.
+
+<LinkButton
+  to="/docs/tutorial/creating-a-source-plugin/part-1/"
+  rightIcon={<MdArrowForward />}
+  variant="SECONDARY"
+>
+  Continue to Part 1
+</LinkButton>

docs/docs/tutorial/creating-a-source-plugin/part-1/index.mdx

@@ -0,0 +1,170 @@
+---
+title: "Part 1: Set up the Project"
+---
+
+import { LinkButton } from "gatsby-interface"
+import Collapsible from "@components/collapsible"
+import { MdArrowForward } from "react-icons/md"
+
+## Introduction
+
+Now that you have the necessary background knowledge and your development environment set up, you can get your local project running!
+
+There are many ways to author a Gatsby plugin or more generally speaking an npm package. Since we don't want you to be stuck in decision fatigue on how to do things, you'll learn how to build a Gatsby source plugin in TypeScript with just `tsc` (that is the TypeScript compiler). Later you can decide to use other tooling if you wish. The repository you'll clone will also show how to test your plugin in a Gatsby site.
+
+By the end of this part of the tutorial, you will have:
+
+- Cloned the companion repository to your local machine
+- Explored the project structure
+- Installed all required dependencies and built all packages
+- Spun up the local development server
+
+## Clone the repository
+
+Clone the [companion repository](https://github.com/gatsbyjs/creating-source-plugin-tutorial) from GitHub to your desired location. The repository's `main` branch shows the finished state after following this tutorial. You can reference its code as an additional resource while going through this guide.
+
+<Collapsible
+  summary={<em>GitHub cloning instructions</em>}
+>
+
+HTTPS snippet:
+
+```shell
+git clone https://github.com/gatsbyjs/creating-source-plugin-tutorial.git
+```
+
+SSH snippet:
+
+```shell
+git clone git@github.com:gatsbyjs/creating-source-plugin-tutorial.git
+```
+
+Also be sure to check out [GitHub's documentation for how to clone a repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository).
+
+</Collapsible>
+
+After successfully cloning the repository, navigate to its folder and switch to the `start` branch to have a clean boilerplate:
+
+```shell
+git checkout start
+```
+
+Open the repository in your favorite editor as it'll make the next step easier.
+
+## Project structure
+
+The `creating-source-plugin-tutorial` project includes three directories:
+
+- `api`: This is the example backend API
+- `plugin`: The source plugin
+- `site`: An example Gatsby site
+
+The `site` uses the `plugin` which sources its data from `api`. As mentioned in [Part 0](/docs/tutorial/creating-a-source-plugin/part-0/#development-environment) this project uses yarn workspaces to make this setup work.
+
+By having the plugin and an example site that uses the plugin in the same repository, you can more easily test your plugin.
+
+<Announcement>
+
+You can ignore the `api` folder unless you're curious how a [GraphQL Yoga server](https://github.com/dotansimha/graphql-yoga) is set up. For the purpose of this tutorial the example backend API was included so that this tutorial doesn't rely on outside resources. This way you can run everything locally.
+
+</Announcement>
+
+The `plugin` itself has a structure we'd also recommend following:
+
+- A `package.json` file that sets `gatsby` and `gatsby-plugin` as its `"keywords"` so that Gatsby's plugin search can pick it up
+- A root `gatsby-node.js` that requires `./dist/gatsby-node` (the compiled TypeScript file). At the moment Gatsby looks for a root `gatsby-node.js` file as its entry file.
+- A well-written `README` that explains to the user in concise steps how to install, use, and configure your plugin (also see [How to write a plugin README](/contributing/docs-contributions/how-to-write-a-plugin-readme/))
+- A `src` folder with the plugin's source code
+  - A root `gatsby-node.ts` file that exports [Gatsby Node APIs](/docs/reference/config-files/gatsby-node/) from individual files
+  - Individual files that only contain the specific Node API (in [kebab case](https://en.wikipedia.org/wiki/Letter_case#Kebab_case)), e.g. `onPluginInit` becomes `on-plugin-init.ts`. This will also make unit testing your plugin easier.
+
+How you organize your `src` folder is of course up to you, but we'd recommend keeping things consistent and files lowercase.
+
+<Collapsible
+  summary={<em>Wondering about a source plugin template for new projects?</em>}
+>
+
+Above we said that you can ignore the `api` folder. This is because for your real-world project you'll most likely source the data from an external API, not an API locally in your repository.
+
+The companion repository not only has a `start` branch but also a `template` branch without the `api` folder. You'll learn more about this at the end in the [What's next?](/docs/tutorial/creating-a-source-plugin/whats-next/) part. For now, go through the tutorial and at the end you'll know how to leverage the repository as a template for your new projects!
+
+</Collapsible>
+
+## Start the project
+
+Ready to see something happen in your browser? Then let's go!
+
+First, install the necessary dependencies to run everything:
+
+```shell
+yarn
+```
+
+Afterwards, open a new window in your terminal (you should have two now).
+
+In the first window, run the `develop:deps` script to start watching the `api` and `plugin` directories. If you change something inside `api`, the GraphQL server will be restarted, if you change something in `plugin` the TypeScript compiler will output updated files.
+
+```shell
+yarn develop:deps
+```
+
+You should find something like this in your terminal:
+
+```shell
+[develop:api   ] 14:48:03 - Starting compilation in watch mode...
+[develop:api   ]
+[develop:plugin] 14:48:03 - Starting compilation in watch mode...
+[develop:plugin]
+[develop:api   ] Server is running at http://localhost:4000/graphql
+[develop:api   ]
+[develop:api   ] 14:48:04 - Found 0 errors. Watching for file changes.
+[develop:plugin]
+[develop:plugin] 14:48:04 - Found 0 errors. Watching for file changes.
+[develop:api   ] Server is running at http://localhost:4000/graphql
+```
+
+Now, in your second terminal window, run the `develop:site` script. This runs `gatsby develop` for the example site:
+
+```shell
+yarn develop:site
+```
+
+While `gatsby develop` is running you should also note the `info Example plugin loaded...` log. This verifies that the plugin was successfully loaded for this initial demo. Yay 🎉
+
+<Announcement>
+
+You're running `develop:site` in a separate window so that you can restart the process independently of the `develop:deps` script. If you make a change to your `plugin` you'll need to restart the example site.
+
+</Announcement>
+
+Once the development server is ready, go to `http://localhost:8000`, and be greeted by a minimalistic index page.
+
+![A screenshot of "localhost:8000" in a web browser. There's a heading that says, "All posts" and a paragraph that says, "Posts will go here"](./01-clean-index-page.png)
+
+### Key takeaways
+
+- It's good practice to develop your source plugin alongside an example site to be able to test it
+- You can keep your code organized by placing each Gatsby Node API in its own file
+- When making changes to your plugin, you'll need to restart `gatsby develop` to see your changes applied
+
+<Announcement>
+
+**Share Your Feedback!**
+
+Our goal is for this tutorial to be helpful and easy to follow. We'd love to hear your feedback about what you liked or didn't like about this part of the tutorial.
+
+Use the "Was this doc helpful to you?" form at the bottom of this page to let us know what worked well and what we can improve.
+
+</Announcement>
+
+### What's coming next?
+
+In Part 2 you'll learn how to do the most important part of a Gatsby source plugin: sourcing data and displaying it on your site.
+
+<LinkButton
+  to="/docs/tutorial/creating-a-source-plugin/part-2/"
+  rightIcon={<MdArrowForward />}
+  variant="SECONDARY"
+>
+  Continue to Part 2
+</LinkButton>