Skip to content

Commit

Permalink
Fix installation steps and broken link in installer/README.md (#473)
Browse files Browse the repository at this point in the history 
* fix broken link and installation steps

* fixed review comments
  • Loading branch information
Amit Roushan authored Feb 3, 2021
1 parent d262a1f commit 8023a1f
Showing 1 changed file with 137 additions and 184 deletions.
321 changes: 137 additions & 184 deletions installer/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,105 +2,59 @@
This is a standalone/non-containerized installer for SODA Infrastructure Manager (delfin) project.
It contains a script and options to check the environment feasible for installing delfin. Installs required dependent software/binaries.

# Prerequisite
Python3 and Pip3 should be installed on the system.
Ensure the logged-in user has root privileges.

Note: If you don't have python3 in your system, you may follow below steps to setup python3 virtual environment.

##### How to setup python3 virtual environment and delfin project

1. Install python-virtual environment package and python3-dev package.

```sh
apt-get install python-virtualenv
apt-get install python3-dev
```
2. Clone delfin repo

```sh
git clone https://github.com/sodafoundation/delfin.git

# Example
root@root1:~/delfin-demo$ git clone https://github.com/sodafoundation/delfin.git

root@root1:~/delfin-demo$ cd delfin
```
# Supported OS
Ubuntu 16.04, Ubuntu 18.04

3. Create a project, using python3
```sh
virtualenv -p /usr/bin/python3.6 project_delfin
# Prerequisite

# Example
root@root1:~/delfin-demo/delfin$ virtualenv -p /usr/bin/python3.6 project_delfin
```
4. Activate your project
- #### Ensure the logged-in user has root privileges.

```sh
source project_delfin/bin/activate
- #### Setup Python3
Python3 and Pip3 should be installed on the system.

# Example
root@root1:~/delfin-demo/delfin$ source project_delfin/bin/activate
# (project_delfin) root@root1:~/delfin-demo/delfin$
```
5. Install all requirements using Pip
Note: If you don't have python3 in your system, you may follow below steps to setup python3 environment.

```sh
pip install -r requirements.txt
```sh
apt-get install python3
apt-get install python3-pip
```

# Example
(project_delfin) root@root1:~/delfin-demo/delfin$ pip install -r requirements.txt
```
6. set PYTHONPATH to working directory
- #### Set PYTHONPATH to working directory

```sh
export PYTHONPATH=$(pwd)
```

```sh
export PYTHONPATH=$(pwd)
```

#### How to setup prometheus to monitor performance metric
- #### Setup Prometheus (for monitor performance metric)

Follow the below steps to setup delfin with prometheus. Once your setup is ready, you can register the storage devices for performance monitoring. Later, the performance metrics can be viewed on prometheus server. This example also guides you to configure and update the targets and interval for scraping the metrics.

Alternatively, you can also watch this [video]((https://drive.google.com/file/d/1WMmLXQeNlToZd0DP5hCFtDZ1IbNJpO6B/view?usp=drivesdk) for more detail.


step1: Install and start prometheus server

1. [Download the latest binaries from here](https://prometheus.io/download/) and run the below steps.


1. tar xvfz prometheus-*.tar.gz

2. cd prometheus-*

3. ./prometheus

Example:
```sh
root@root:/prometheus/prometheus-2.20.0.linux-amd64$ ./prometheus
```

2. Edit the prometheus.yml and set the appropriate target, interval and metrics_api path. Below is sample example of prometheus.yml

###### prometheus.yml

global:
scrape_interval: 900s
scrape_configs:
- job_name: delfin-prometheus
metrics_path: /metrics
static_configs:
- targets:
- localhost:8195

# Supported OS
Ubuntu 16.04, Ubuntu 18.04

# Logs
All the installer logs are stored in the /var/log/sodafoundation directory.
The logs can be uniquely identified based upon the timestamp.

Alternatively, you can also watch this [video](https://drive.google.com/file/d/1WMmLXQeNlToZd0DP5hCFtDZ1IbNJpO6B/view?usp=drivesdk) for more detail.

[Download the latest binaries from here](https://prometheus.io/download/) and run the below steps.

1. tar xvfz prometheus-*.tar.gz

2. cd prometheus-*
3. Edit the prometheus.yml and set the appropriate target, interval and metrics_api path.
Below is sample example of prometheus.yml
###### prometheus.yml
```
global:
scrape_interval: 900s
scrape_configs:
- job_name: delfin-prometheus
metrics_path: /metrics
static_configs:
- targets:
- 'localhost:8195'
```
4. ./prometheus

Example:
```sh
root@root:/prometheus/prometheus-2.20.0.linux-amd64$ ./prometheus
```
# Structure of the installer
This installer comes with options of pre-check, install and uninstall
pre-check: This script checks for the components required by delfin to function. If they are not present, precheck will install them.
Expand All @@ -115,36 +69,36 @@ Install script can be executed with three different switches to:
- only run the installer without doing pre-check (if pre-check has been executed explicitly) [./install -s]
- execute pre-check as well the install [./install]
## For the available options for install, you can execute 'install -h'
```sh
installer/install -h

# Example
root@root1:~/delfin-demo/delfin$ installer/install -h

Usage install [--help|--precheck|--skip_precheck]
Usage:
install [-h|--help]
install [-p|--precheck]
install [-s|--skip_precheck]
Flags:
-h, --help Print the usage of install
-p, --precheck Only perform system software requirements for installation
-s, --skip_precheck If precheck is not required and directly install
```
#### For the available options for install, you can execute 'install -h'
```sh
installer/install -h
# Example
root@root1:~/delfin-demo/delfin$ installer/install -h
Usage install [--help|--precheck|--skip_precheck]
Usage:
install [-h|--help]
install [-p|--precheck]
install [-s|--skip_precheck]
Flags:
-h, --help Print the usage of install
-p, --precheck Only perform system software requirements for installation
-s, --skip_precheck If precheck is not required and directly install
```
## For Pre-check, run below command
```sh
installer/install -p
#### For Pre-check, run below command
```sh
installer/install -p
# Example
# Example
root@root1:~/delfin-demo/delfin$ installer/install -p
OR
root@root1:~/delfin-demo/delfin/installer$ ./install --precheck
```
root@root1:~/delfin-demo/delfin$ installer/install -p
OR
root@root1:~/delfin-demo/delfin/installer$ ./install --precheck
```
## Install without pre-check
#### Install without pre-check
```sh
installer/install -s
Expand All @@ -153,7 +107,7 @@ installer/install -s
root@root1:~/delfin-demo/delfin$ installer/install -s
```
## Execute both pre-check as well as install
#### Execute install with precheck
```sh
installer/install
Expand All @@ -163,81 +117,80 @@ root@root1:~/delfin-demo/delfin$ installer/install
# Uninstall
Running the uninstall script will stop all delfin processes and do cleanup

installer/uninstall

# Example
root@root1:~/delfin-demo/delfin$ installer/uninstall


# Test the running delfin setup/process
1. Make sure all delfin process are up and running

```
ps -ef|grep delfin
# Example
root@root1:~/delfin-demo/delfin# ps -ef |grep delfin
root 25856 3570 0 00:21 pts/0 00:00:04 python3 /root/delfin-demo/delfin/installer/../delfin/cmd/api.py --config-file /etc/delfin/delfin.conf
root 25858 3570 0 00:21 pts/0 00:00:09 python3 /root/delfin-demo/delfin/installer/../delfin/cmd/task.py --config-file /etc/delfin/delfin.conf
root 25860 3570 0 00:21 pts/0 00:00:06 python3 /root/delfin-demo/delfin/installer/../delfin/cmd/alert.py --config-file /etc/delfin/delfin.conf
root 25862 3570 0 00:21 pts/0 00:00:00 python3 /root/delfin-demo/delfin/installer/../delfin/exporter/exporter_server.py --config-file /etc/delfin/delfin.conf
```sh
installer/uninstall
```
# Example
root@root1:~/delfin-demo/delfin$ installer/uninstall
```
2. Register storages
# Logs
All the installer logs are stored in the /var/log/sodafoundation directory.
The logs can be uniquely identified based upon the timestamp.
POST http://localhost:8190/v1/storages
body :
```
{
"vendor": "fake_storage",
"model": "fake_driver",
"rest": {
"host": "127.0.0.1",
"port": 8088,
"username": "admin",
"password": "pass"
# Test the running delfin setup/process
1. Make sure all delfin process are up and running
```
ps -ef|grep delfin
# Example
root@root1:~/delfin-demo/delfin# ps -ef |grep delfin
root 25856 3570 0 00:21 pts/0 00:00:04 python3 /root/delfin-demo/delfin/installer/../delfin/cmd/api.py --config-file /etc/delfin/delfin.conf
root 25858 3570 0 00:21 pts/0 00:00:09 python3 /root/delfin-demo/delfin/installer/../delfin/cmd/task.py --config-file /etc/delfin/delfin.conf
root 25860 3570 0 00:21 pts/0 00:00:06 python3 /root/delfin-demo/delfin/installer/../delfin/cmd/alert.py --config-file /etc/delfin/delfin.conf
root 25862 3570 0 00:21 pts/0 00:00:00 python3 /root/delfin-demo/delfin/installer/../delfin/exporter/exporter_server.py --config-file /etc/delfin/delfin.conf
```
2. Register storages
POST http://localhost:8190/v1/storages
body :
```
{
"vendor":"fake_storage",
"model":"fake_driver",
"rest":{
"host":"127.0.0.1",
"port":8088,
"username":"admin",
"password":"pass"
},
"extra_attributes": {
"array_id": "12345"
"extra_attributes":{
"array_id":"12345"
}
}
```
3. Run the GET API to get the registered storages.
}
```
3. Run the GET API to get the registered storages.
GET http://localhost:8190/v1/storages
GET http://localhost:8190/v1/storages
use storage_id for registering storage for performance collection or alert monitoring
4. Register storage for performance collection
a. There are two exporters supported as of now for performance collection, prometheus and kafka. To enable the exporters,
environment variable need to set as True. Whichever value is set as True, will be enabled for collection
Ex:
export KAFKA=True
export PROMETHEUS=True
b. Use API to register the storages for performance collection
PUT http://localhost:8190/v1/storages/<storage_id>/metrics-config
use storage_id for registering storage for performance collection or alert monitoring
4. Register storage for performance collection
a. There are two exporters supported as of now for performance collection, prometheus and kafka. Uncomment performance_exporters option in <delfin_path>/etc/delffin/delfin.conf as below to enable exporter for prometheus and kafka
```
Ex:
# Uncomment or add exporters
performance_exporters = PerformanceExporterPrometheus, PerformanceExporterKafka
```
b. Use API to register the storages for performance collection
PUT http://localhost:8190/v1/storages/<storage_id>/metrics-config
body:
```
{
"array_polling":{
"perf_collection":true,
"interval":900,
"is_historic":true
}
}
```
5. Monitor the performance metrics on prometheus server at default location
body:
```
{
"array_polling": {
"perf_collection": true,
"interval": 900,
"is_historic": true
}
}
```
5. Monitor the performance metrics on prometheus server at default location

http://localhost:9090/graph
http://localhost:9090/graph

0 comments on commit 8023a1f

Please sign in to comment.