Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: developer documentation #2275

Merged
merged 4 commits into from
Jan 9, 2023
Merged

docs: developer documentation #2275

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d9fa0fd
docs: developer documentation (#2165) (#2275)
cramakri Jan 5, 2023
3f7302e
responding to change requests
cramakri Jan 6, 2023
169cea7
add some troubleshooting documentation
cramakri Jan 6, 2023
9a8b39c
Merge branch 'master' into 2165-dev-docs
lorenzo-cavazzi Jan 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
250 changes: 205 additions & 45 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,77 +2,237 @@
[![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg?style=flat-square)](https://conventionalcommits.org)
[![Release](https://img.shields.io/github/tag/SwissDataScienceCenter/renku-ui.svg)](https://github.com/SwissDataScienceCenter/renku-ui/releases)

# Renku-UI
# Renku UI

## Quickstart
The web-based UI for [Renku](https://github.com/SwissDataScienceCenter/renku).

The Renku ui depends on a running instance of Renku being present. More
precisely, it relies on a running instance of the [Renku
Gateway](https://github.com/SwissDataScienceCenter/renku-gateway) that
acts as an interface to all backend services APIs, handling
authentication and exchanging access tokens. Follow these
[instructions](https://renku.readthedocs.io/en/latest/how-to-guides/admin/deploying-renku.html)
to get Renku up and running. You can also deploy an environment in a
remote development cluster.
This README contains information relevant to contributors to the renku-ui component. To deploy RenkuLab, including the UI, see the [Administrator's Guide](https://renku.readthedocs.io/en/latest/how-to-guides/admin/deploying-renku.html).

## Developing the UI
# Architecture

Once you have a development instance of Renku running locally or in the
cloud, you can install
[telepresence](https://www.telepresence.io/reference/install) locally
and run the `run-telepresence.sh` script in the client or
the server folder. Don't forget to run install before running telepresence for the first time or
after any package change.
The Renku UI is made up of two subcomponents, which are each packaged as [Docker](https://www.docker.com) containers and are deployed using [Kubernetes](https://kubernetes.io).

| | |
|--------|-------------------------------------------------|
| client | [React-based](https://reactjs.org) front-end |
| server | [Express-based](https://expressjs.com) back-end |

Below, we explain each of those components in greater detail and guidelines for developing code within them.

# Prerequisites

To develop on this codebase, you will want the following tools installed.


| | |
|----------------------------------------------------------------------|---------------------------------------------------------------------|
| [docker](https://www.docker.com) | For building containers |
| [helm](https://helm.sh) | For packaging things for K8s |
| [kubectl](https://kubernetes.io/docs/tasks/tools/) | K8s command-line tool |
| [nvm](https://github.com/nvm-sh/nvm) | NVM or some similar tool for managing node environments |
| [telepresence](https://www.telepresence.io/docs/latest/quick-start/) | Tool for redirecting requests to your local development environment |

You should use your **node version manager** to install the node version specified in the [Dockerfile](https://github.com/SwissDataScienceCenter/renku-ui/blob/master/client/Dockerfile).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of these tools are not required for developing with Cypress on the client. And I believe helm is not needed in most cases, just for deploying to an own namespace but not for interacting with CI deployments.
Should we rephrase to indicate that npm is always necessary, and docker+kubectl are required for telepresence?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion, I have split up the prerequisites into three sections -- always needed, needed for telepresence, and needed to deploy to K8s.



# Client

The client is the [React-based](https://reactjs.org) front-end for RenkuLab. Development started in 2017, and, in the intervening years, we have striven to change our development style to reflect the evolving best practices around the tools we use. You will see that not all code conforms to the guidelines laid out in this document, but **new code** should follow these guidelines, and older code should be refactored at opportune moments, whenever possible, to conform as well.


## Tool Stack

| Framework | Purpose |
|-----------------------------------------------|-----------------------------------------------|
| [Bootstrap](https://getbootstrap.com) | Responsive grid, HTML styling, and components |
| [Cypress](https://www.cypress.io) | UI testing framework |
| [React](https://reactjs.org) | Reactive component framework |
| [Reactstrap](https://reactstrap.github.io/) | React bindings for Bootstrap |
| [Redux Toolkit](https://redux-toolkit.js.org) | State and React-integration components |
| [Storybook](https://storybook.js.org) | UI component documentation |
| [TypeScript](https://www.typescriptlang.org) | JavaScript extension with static typing |


## Unit Tests and Linting

We use [jest](https://jestjs.io) as for unit tests and
[eslint](https://eslint.org/) to detect trivial errors and enforce some coding-style preferences. We require both
commands to terminate without warnings before we merge a PR. You can
manually run tests using the following commands:

$ cd client # or server if you need to work there
$ npm test
$ npm run lint

Some linting errors can be automatically fixed by running
`npm run lint-fix`.

We suggest using an IDE that supports eslint (like [VS Code](https://code.visualstudio.com) or similar) and configuring your IDE to integrate with our eslint configuration so that any linting errors will be displayed as you develop rather than waiting for the CI pipeline to flag them.

## Developing

As you would expect for a React-based project, you can install the client using `npm`.

$ cd client
$ npm install

This will install the toolchain for developing the client as well as any libraries used in the UI.

To run and interact with the UI code on your development machine, you can either use the Cypress tests, which provide mocked responses from the backend, or use telepresence to replace the UI client component in a K8s-based deployment with the component running on your machine.

### Cypress

We have an ever-growing suite of UI tests developed with Cypress. The tests can be run with mock responses from the backend, and can therefore be used to try out changes to the UI without access to a development cluster.

$ cd tests
$ npm install
$ npm run e2e:local

### Telepresence

Telepresence can be used to develop the UI in a realistic setting. The client folder includes a `run-telepresence.sh` script that is tailored for the SDSC development cluster.

$ cd client
$ ./run-telepresence.sh

Telepresence replaces the selected UI pod in the target Kubernetes
instance. All the traffic is then redirected to a local process, making
all the changes to files almost immediately available in your
development RenkuLab instance.
Telepresence replaces the UI client pod in the target Kubernetes instance. All the traffic is then redirected to a local process, making changes to files almost immediately available in your development RenkuLab instance.

The `run-telepresence.sh` script uses the current K8s context as returned by `kubectl config current-context`. You can deploy your own RenkuLab application (using the helm chart in the renku repository) for development; alternatively the renku-ui repository includes CI tasks that can deploy the code for a PR. To take advantage of this task, add

The `run-telepresence.sh` scripts support out-of-the-box
[telepresence](https://www.telepresence.io/reference/install) minikube
and the Renku team [switch-dev]{.title-ref} cloud. You need to properly
set the environment variable `CURRENT_CONTEXT` to either `"minikube"` or
`"switch-dev"`.
```
/deploy #persist
```

To the body of your PR description.

There are a few other environment variables you may want to set when
starting telepresence if you are going to to take advantage of the Renku
team internal development infrastructure:
There are a few environment variables you may want to set when starting telepresence if you are going to to take advantage of the Renku team internal development infrastructure:

- SENTRY: set to 1 to redirect the exceptions to the dev
[sentry](https://sentry.dev.renku.ch) deployment
- PR: set to the target PR number in the
[renku-ui](https://github.com/SwissDataScienceCenter/renku-ui/pulls)
repo to work in the corresponding CI deployment
- SENTRY: set to 1 to redirect the exceptions to the dev [sentry](https://sentry.dev.renku.ch) deployment
- PR: set to the target PR number in the [renku-ui](https://github.com/SwissDataScienceCenter/renku-ui/pulls) repo to work in the corresponding CI deployment
- DEV_NAMESPACE: if you do not target a PR deployment you should specify a value for this to reference the namespace with your development deployment


For example:
```
$ SENTRY=0 PR=1166 ./run-telepresence.sh
```
There are other variables used in the `run-telepresence.sh` script. For
specific use cases, you may want to modify some values manually.

## Tests
There are also further configuration possibilities offered by the `run-telepresence.sh` script. For
specific use cases, you may need to modify the script directly, since not all options are configurable through environment variables.

## Coding Guidelines

As explained above, not all the code in the repo conforms to these guidelines, but this section explains how code _should_ be developed in the repo.

We use [jest](https://jestjs.io) as our default testing framework and
[eslint](https://eslint.org/) as linter. Mind that we require both
### **Use Typescript**

New code should be developed in TypeScript, and older code should be converted to TypeScript whenever feasible.

### **React components should be functional and take advantage of hooks**

* Use `useState` to manage component-local state
* Use `useEffect` for handling the component lifecycle

### **Application state is managed by Redux**

We follow the [redux-toolkit](https://redux-toolkit.js.org/tutorials/typescript) style for interacting with Redux. If you are not familiar with this, follow the link to see the tutorial.

Key points around this include:

* Global application state is kept in a single, global [Redux](https://redux.js.org) store
* Features (see below) should make [slices](https://redux-toolkit.js.org/usage/usage-with-typescript#createslice) that encapsulate the state they need and add the slices into the [global store](https://redux-toolkit.js.org/api/configureStore).
* Use the `useSelector` hook to access information from slices in components
* Use the `useDispatch` hook to make changes to state in components

### **Interact with backend services using RTK Query**

[RTK Query](https://redux-toolkit.js.org/tutorials/rtk-query) greatly simplifies interacting with other services. It automatically provides integration with the Redux store and handles caching of responses and transitions through the request lifecycle (e.g., _fetching_, etc.).

### **Group shared code into features**

[Features](https://redux.js.org/faq/code-structure) should be used to group together UI components and state manipulation code.

It commonly happens that there are multiple components and multiple state interactions that center around one shared slice and/or one backend API. These should be grouped into a feature to make this structure easier to work with.

# Server

The server is the [Express-based](https://expressjs.com) back-end for the RenkuLab UI. At first, the front-end would interact directly with back-end services, but this structure imposed some limitations and complications, so we introduced a server component with the sole responsibility of serving the UI and simplifying interactions with the other backend services.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove the historical reasons for creating the server, and mention that it also handles requesting, storing, and renewing user tokens because it's safer than doing it in the client and it allows us to poll some resources on behalf of the user

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. I have made these changes.


In many cases, the server just forwards requests to the appropriate service, and is therefore a much smaller and simpler codebase than the client. Where appropriate, though, the server can be used to implement logic or interact with multiple services to provide a more unified view to the client. The server manages websockets to send asynchronous notifications to clients when important events occur.

## Tool Stack

| Framework | Purpose |
|----------------------------------------------------------------------|-----------------------------------------------|
| [Express](https://expressjs.com) | Route and respond to HTTP requests |
| [Morgan](https://expressjs.com/en/resources/middleware/morgan.html) | Logging middleware for express |
| [TypeScript](https://www.typescriptlang.org) | JavaScript extension with static typing |
| [WS](https://www.npmjs.com/package/ws) | WebSocket framework |


## Unit Tests and Linting

As with the client, we use [jest](https://jestjs.io) as for unit tests and
[eslint](https://eslint.org/) to detect trivial errors and enforce some coding-style preferences. We require both
commands to terminate without warnings before we merge a PR. You can
manually run tests using the following commands:

$ cd client # or server if you need to work there
$ cd server
$ npm test
$ npm run lint

Some linting errors can be automatically fixed by running
`npm run lint-fix`. We suggest using an IDE that supports eslint (like
[vscode](https://code.visualstudio.com) or similar) to get realtime
feedback when modifying the code.
`npm run lint-fix`.

We suggest using an IDE that supports eslint (like [VS Code](https://code.visualstudio.com) or similar) and configuring your IDE to integrate with our eslint configuration so that any linting errors will be displayed as you develop rather than waiting for the CI pipeline to flag them.

## Developing

You can install the server using `npm`.

$ cd server
$ npm install

This will install the toolchain for developing the server as well as any libraries used.

To run and interact with the server code on your development machine, you will need to use telepresence to replace the UI server component in a K8s-based deployment with the component running on your machine.

### Telepresence

The server folder includes a `run-telepresence.sh` script that is tailored for the SDSC development cluster.

$ cd server
$ ./run-telepresence.sh

Telepresence replaces the UI server pod in the target Kubernetes instance. All the traffic is then redirected to a local process, making changes to files almost immediately available in your development RenkuLab instance.

The instructions for deploying the renku application into K8s are the same as in the client, so see that section for those details. There are some small differences in how the server `run-telepresence.sh` script works compared to the client.

* In the server, `run-telepresence.sh` will prompt you to provide or override the deployment you want to intercept. You can enter the id of a PR to intercept the deployment associated with a GitHub PR.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm tempted to adjust the 2 telepresence scripts to unify this behavior.
If it's ok with you, once the PR is ready I would add a commit to align it (and update this section accordingly) .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I'm ok with that change, but I think it would be nicer if the prompting only happened if no values were provided. That is, if there is no DEV_NAMESPACE or PR environment variable set, then prompt for it, but one of them is set, do not prompt and just use what was provided.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion! I will go that way 👍

* By default, the script will start the server and wait for a debugger to attach. Instead, you can start in _console mode_ and start the server yourself by running `npm run dev-debug` (within a debugger).

For the default mode of the script, you will need to attach a debugger to finish bringing the server up. In VS Code, you can add a **Node Attach** run configuration for this. It will look something like:

```
{
"version": "0.2.0",
"configurations": [
{
"name": "uiserver",
"type": "node",
"request": "attach",
"address": "localhost",
"port": 9229,
"protocol": "inspector",
"restart": true
}
]
}
```

With this configuration, after running `run-telepresence.sh` you should invoke the _Run > Start Debugging_ in VS Code to attach the debugger.

## Site navigation map
# Site navigation map
Note: stroke-dasharray when the link is only for anonymous users

```mermaid
Expand All @@ -94,7 +254,7 @@ flowchart LR
DA-->DAID(/:id)
end
subgraph L3
PRID-->PRID1(/overview/stats)
PRID-->PRID1(/overview/stats)
PRID-->PRID2(/overview/commits)
PRID-->PRID3(/overview/status)
PRID-->PRID4(/collaboration/issues)
Expand Down