feat: RedwoodJS Studio & OpenTelemetry (Dev)

[Josh-Walker-GM]

RedwoodJS Studio [Experimental]

This PR adds the initial experimental version of a "RedwoodJS Studio". This is a CLI command yarn rw experimental studio which spawns a development studio which users can use to gain insights into their project via the OpenTelemetry instrumentation.

For documentation and discussion please see the forum post:
https://community.redwoodjs.com/t/redwood-studio-experimental/4771

Features

• Tracing page which lists all the ingested OpenTelemetry traces and trace visualisation via three modes; timeline, flame table and prisma queries
• SQL page which exposes a list of all the queries/statements made
• GraphiQL page which allows access to the standard GraphiQL interface but also is able to support basic user impersonation.

Setup / Requirements
There is not explicit setup of the studio required. You will need to have setup OpenTelemetry to enable the studio to gather the data it uses to provide you with insights into your project. The studio accepts OpenTelemetry tracing data at http://localhost:4318/v1/traces.

Config
We have adding some optional settings to the redwood.toml file. See below for the options added with descriptions of their purpose.

[experimental.studio]
  # Determines whether the studio should run with an in memory database or persist the data
  # to a file in your project within `./redwood` 
  inMemory = false

[experimental.studio.graphiql]
  endpoint = 'graphql'

[experimental.studio.graphiql.authImpersonation]
  # authProvider = undefined (default value)
  jwtSecret = 'secret'
  # userId = undefined (default value)
  # email = undefined (default value)
  # roles = undefined (default value)

OpenTelemetry Support [Experimental]

This PR also adds support for OpenTelemetry instrumentation of components of the framework. It exposes a setup command which generated the boilerplate needed to gather instrumentation from the API side of a redwood project: yarn rw experimental setup opentelemetry

⚠️ - This is currently only supported during development. Enabling OpenTelemetry in production will be our next work after this but will require other structural changes to the framework/deployments.

For documentation and discussion please see the forum post:
https://community.redwoodjs.com/t/opentelemetry-support-experimental/4772

Features

• Sets up an OpenTelemetry sdk file which should provide a boilerplate which enables gathering tracing from your project
• The graphql yoga server now is instrumented with OpenTelemetry via an envelop plugin
• API functions are automatically wrapped to produce OpenTelemetry spans via a babel plugin

Setup / Requirements
yarn rw experimental setup opentelemetry should perform all the necessary setup needed to enable gathering OpenTelemetry traces from your project.

This will add the necessary dependencies to your project and produce an opentelemetry.ts|js file within your API side. This file is where your SDK is setup to collect and export the telemetry generated by your Redwood app. By default your telemetry will be exported to http://localhost:4318/v1/traces you can change to suit your needs. This is the location that Redwood Studio will listen for traces so leave this as the default if you wish to use the studio.

Config
You should be able to configure the behaviour of OpenTelemetry from within the opentelemetry.ts|js file which was generated. There is one additional TOML field which will be setup which is used to inform Redwood of where this setup file lives:

[experimental.opentelemetry]
  enabled = true
  apiSdk = "/home/linux-user/redwood-project/api/src/opentelemetry.js"

---

Pull Request - Outstanding Work

1. Avoid CORS setup needed by graphiql by using an API proxy on the studio backend.
2. Conditionally enable the opentelemetry wrapping babel plugin - only if user wants/has otel enabled.
3. Fix/rework the graphql-server resolver wrapping.
4. Documentation - See forum posts for documentation of experimental features
   1. RedwoodJS Studio
   2. OpenTelemetry setup

Ping: @dthyresson

[Josh-Walker-GM]

Great work on this @Josh-Walker-GM 🚀

This needs to be clearly marked as Experimental: we are not guaranteering it will become a feature and we need (if applicable) to break things outside of semver releases.

Can you please update the README to indicate it's experimental and the brief rationale above?

Also, do you think we can follow the pattern for TOML we discussed, so something like [experimental.studio]? (And/or were you planning to do that?) Just confirming as I understand this README is documentation.

Hey @thedavidprice, yeah I made the following changes which should have hopefully moved this in the direction you requested:

1. TOML options are now prefixed with experimental so they look like the following:

[experimental.opentelemetry]
  enabled = true
  apiSdk = "/home/josh/Development/redwood-test-project/api/src/opentelemetry.js"

2. Commands now have an experimental prefix so they look like the following:

experimental-command-demo.mp4

This should provide the following benefits:

1. It is clear to the user which TOML is for experimental features. We can state clearly that any TOML with an experimental prefix does not follow SEMVER and can/will break, change or be deprecated often.
2. Experimental CLI commands are not only obviously experimental given the prefix but are also discoverable in the standard yargs way - i.e. providing no command will list the available commands as shown in the demo above.

---

I will update the readme to be explicit in its "experimental" status and will also add links to forum pages for both CLI commands.

Sound/look good?

[thedavidprice]

@Josh-Walker-GM Yes to all the above! Screen recording is exciting 🚀

My only 2¢ is to provide an alias for the yarn rw experimental [command] so users could type in something like yarn rw exp [command]. I defer to you.

Plan == approved.

[Josh-Walker-GM]

Alias is a good idea! Added it in as exp