diff --git a/docs/provisioning.md b/docs/platform-management.md similarity index 83% rename from docs/provisioning.md rename to docs/platform-management.md index fb9c6edb..28a8b407 100644 --- a/docs/provisioning.md +++ b/docs/platform-management.md @@ -1,5 +1,4 @@ -Provisioning is a process of configuration of an IoT platform in which system operator creates and sets-up different entities -used in the platform - users, channels and things. +Provisioning is a process of configuration of an IoT platform in which system operator creates and sets-up different entities used in the platform - users, channels, and things. ## Users management @@ -11,20 +10,18 @@ Use the Mainflux API to create user account: curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" https://localhost/users -d '{"email":"john.doe@email.com", "password":"12345678"}' ``` -Note that when using official `docker-compose`, all services are behind `nginx` -proxy and all traffic is `TLS` encrypted. +Note that when using official `docker-compose`, all services are behind `nginx` proxy and all traffic is `TLS` encrypted. ### Obtaining an authorization key -In order for this user to be able to authenticate to the system, you will have -to create an authorization token for him: +In order for this user to be able to authenticate to the system, you will have to create an authorization token for him: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" https://localhost/tokens -d '{"email":"john.doe@email.com", "password":"12345678"}' ``` Response should look like this: -``` +```json { "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1MjMzODg0NzcsImlhdCI6MTUyMzM1MjQ3NywiaXNzIjoibWFpbmZsdXgiLCJzdWIiOiJqb2huLmRvZUBlbWFpbC5jb20ifQ.cygz9zoqD7Rd8f88hpQNilTCAS1DrLLgLg4PRcH-iAI" } @@ -32,23 +29,19 @@ Response should look like this: ## System provisioning -Before proceeding, make sure that you have created a new account, and obtained -an authorization key. +Before proceeding, make sure that you have created a new account, and obtained an authorization key. ### Provisioning things > This endpoint will be depreciated in 0.11.0. It will be replaced with the bulk endpoint currently found at /things/bulk. -Things are created by executing request `POST /things` with a JSON payload. -Note that you will also need `user_auth_token` in order to create things -that belong to this particular user. +Things are created by executing request `POST /things` with a JSON payload. Note that you will also need `user_auth_token` in order to create things that belong to this particular user. -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" -H "Authorization: " https://localhost/things -d '{"name":"weio"}' ``` -Response will contain `Location` header whose value represents path to newly -created thing: +Response will contain `Location` header whose value represents path to newly created thing: ``` HTTP/1.1 201 Created @@ -68,7 +61,7 @@ curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: applica The response's body will contain a list of the created things. -``` +```json HTTP/2 201 server: nginx/1.16.0 date: Tue, 22 Oct 2019 02:19:15 GMT @@ -81,17 +74,15 @@ access-control-expose-headers: Location ### Retrieving provisioned things -In order to retrieve data of provisioned things that is written in database, you -can send following request: +In order to retrieve data of provisioned things that is written in database, you can send following request: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: " https://localhost/things ``` -Notice that you will receive only those things that were provisioned by -`user_auth_token` owner. +Notice that you will receive only those things that were provisioned by `user_auth_token` owner. -``` +```json HTTP/1.1 200 OK Content-Type: application/json Date: Tue, 10 Apr 2018 10:50:12 GMT @@ -116,29 +107,25 @@ Content-Length: 1105 } ``` -You can specify `offset` and `limit` parameters in order to fetch specific -group of things. In that case, your request should look like: +You can specify `offset` and `limit` parameters in order to fetch a specific group of things. In that case, your request should look like: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: " https://localhost/things?offset=0&limit=5 ``` -You can specify `name` and/or `metadata` parameters in order to fetch specific -group of things. When specifiying metadata you can specify just a part of the metadata json you want to match +You can specify `name` and/or `metadata` parameters in order to fetch specific group of things. When specifying metadata you can specify just a part of the metadata JSON you want to match. -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: " https://localhost/things?offset=0&limit=5&metadata={"serial":"123456"} ``` -If you don't provide them, default values will be used instead: 0 for `offset`, -and 10 for `limit`. Note that `limit` cannot be set to values greater than 100. Providing -invalid values will be considered malformed request. +If you don't provide them, default values will be used instead: 0 for `offset`, and 10 for `limit`. Note that `limit` cannot be set to values greater than 100. Providing invalid values will be considered malformed request. ### Removing things In order to remove you own thing you can send following request: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -X DELETE -H "Authorization: " https://localhost/things/ ``` @@ -152,8 +139,7 @@ Channels are created by executing request `POST /channels`: curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: application/json" -H "Authorization: " https://localhost/channels -d '{"name":"mychan"}' ``` -After sending request you should receive response with `Location` header that -contains path to newly created channel: +After sending request you should receive response with `Location` header that contains path to newly created channel: ``` HTTP/1.1 201 Created @@ -173,7 +159,7 @@ curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: applica The response's body will contain a list of the created channels. -``` +```json HTTP/2 201 server: nginx/1.16.0 date: Tue, 22 Oct 2019 02:14:41 GMT @@ -186,17 +172,15 @@ access-control-expose-headers: Location ### Retrieving provisioned channels -To retreve provisioned channels you should send request to `/channels` with -authorization token in `Authorization` header: +To retreve provisioned channels you should send request to `/channels` with authorization token in `Authorization` header: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: " https://localhost/channels ``` -Note that you will receive only those channels that were created by authorization -token's owner. +Note that you will receive only those channels that were created by authorization token's owner. -``` +```json HTTP/1.1 200 OK Content-Type: application/json Date: Tue, 10 Apr 2018 11:38:06 GMT @@ -215,35 +199,28 @@ Content-Length: 139 } ``` -You can specify `offset` and `limit` parameters in order to fetch specific -group of channels. In that case, your request should look like: +You can specify `offset` and `limit` parameters in order to fetch specific group of channels. In that case, your request should look like: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: " https://localhost/channels?offset=0&limit=5 ``` -If you don't provide them, default values will be used instead: 0 for `offset`, -and 10 for `limit`. Note that `limit` cannot be set to values greater than 100. Providing -invalid values will be considered malformed request. +If you don't provide them, default values will be used instead: 0 for `offset`, and 10 for `limit`. Note that `limit` cannot be set to values greater than 100. Providing invalid values will be considered malformed request. ### Removing channels In order to remove specific channel you should send following request: -``` +``` bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -X DELETE -H "Authorization: " https://localhost/channels/ ``` ## Access control -Channel can be observed as a communication group of things. Only things that -are connected to the channel can send and receive messages from other things -in this channel. things that are not connected to this channel are not allowed -to communicate over it. +Channel can be observed as a communication group of things. Only things that are connected to the channel can send and receive messages from other things in this channel. +Things that are not connected to this channel are not allowed to communicate over it. -Only user, who is the owner of a channel and of the things, can connect the -things to the channel (which is equivalent of giving permissions to these things -to communicate over given communication group). +Only user, who is the owner of a channel and of the things, can connect the things to the channel (which is equivalent of giving permissions to these things to communicate over given communication group). To connect a thing to the channel you should send following request: @@ -261,13 +238,13 @@ curl -s -S -i --cacert docker/ssl/certs/ca.crt -X POST -H "Content-Type: applica You can observe which things are connected to specific channel: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: " https://localhost/channels//things ``` Response that you'll get should look like this: -``` +```json { "total": 2, "offset": 0, @@ -291,13 +268,13 @@ Response that you'll get should look like this: You can also observe to which channels is specified thing connected: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -H "Authorization: " https://localhost/things//channels ``` Response that you'll get should look like this: -``` +```json { "total": 2, "offset": 0, @@ -319,6 +296,6 @@ Response that you'll get should look like this: If you want to disconnect your thing from the channel, send following request: -``` +```bash curl -s -S -i --cacert docker/ssl/certs/ca.crt -X DELETE -H "Authorization: " https://localhost/channels//things/ ``` diff --git a/docs/provision.md b/docs/provision.md new file mode 100644 index 00000000..092450fb --- /dev/null +++ b/docs/provision.md @@ -0,0 +1,156 @@ +# Provision service + +Provisioning is a process of configuration of an IoT platform in which system operator creates and sets-up different entities used in the platform - users, channels, and things. It is part of process of setting up IoT applications where we connect devices on edge with platform in cloud. + +For provisioning we can use [Mainflux CLI][cli] for creating users and for each node in the edge (eg. gateway) required number of things, channels, connecting them, and creating certificates if needed. + +Provision service is used to set up initial application configuration once user is created. Provision service creates things, channels, connections, and certificates. Once user is created we can use provision to create a setup for edge node in one HTTP request instead of issuing several CLI commands. + +Provision service provides an HTTP API to interact with [Mainflux][mainflux]. + +For gateways to communicate with [Mainflux][mainflux] configuration is required (MQTT host, thing, channels, certificates...). Gateway will send a request to [Bootstrap][bootstrap] service providing `` and `` in HTTP request to get the configuration. To make a request to [Bootstrap][bootstrap] service you can use [Agent][agent] service on a gateway. + +To create bootstrap configuration you can use [Bootstrap][bootstrap] or `Provision` service. [Mainflux UI][mfxui] uses [Bootstrap][bootstrap] service for creating gateway configurations. `Provision` service should provide an easy way of provisioning your gateways i.e creating bootstrap configuration and as many things and channels that your setup requires. + +Also, you may use provision service to create certificates for each thing. Each service running on gateway may require more than one thing and channel for communication. +If, for example, you are using services [Agent][agent] and [Export][exp] on a gateway you will need two channels for `Agent` (`data` and `control`) and one thing for `Export`. +Additionally, if you enabled mTLS each service will need its own thing and certificate for access to [Mainflux][mainflux]. +Your setup could require any number of things and channels, this kind of setup we can call `provision layout`. + +Provision service provides a way of specifying this `provision layout` and creating a setup according to that layout by serving requests on `/mapping` endpoint. Provision layout is configured in [config.toml](configs/config.toml). + +## Configuration + +The service is configured using the environment variables presented in the following [table][config]. Note that any unset variables will be replaced with their default values. + + +By default, call to `/mapping` endpoint will create one thing and two channels (`control` and `data`) and connect it. If there is a requirement for different provision layout we can use [config][conftoml] file in addition to environment variables. + +For the purposes of running provision as an add-on in docker composition environment variables seems more suitable. Environment variables are set in [.env][env]. + +Configuration can be specified in [config.toml][conftoml]. Config file can specify all the settings that environment variables can configure and in addition `/mapping` endpoint provision layout can be configured. + +In `config.toml` we can enlist an array of things and channels that we want to create and make connections between them which we call provision layout. + +Things Metadata can be whatever suits your needs. Thing that has metadata with `external_id` will have bootstrap configuration created, `external_id` value will be populated with value from [request](#example)). +Bootstrap configuration can be fetched with [Agent][agent]. For channel's metadata `type` is reserved for `control` and `data` which we use with [Agent][agent]. + +Example of provision layout below +```toml +[[things]] + name = "thing" + + [things.metadata] + external_id = "xxxxxx" + + +[[channels]] + name = "control-channel" + + [channels.metadata] + type = "control" + +[[channels]] + name = "data-channel" + + [channels.metadata] + type = "data" + +[[channels]] + name = "export-channel" + + [channels.metadata] + type = "data" +``` + +## Authentication +In order to create necessary entities provision service needs to authenticate against Mainflux. +To provide authentication credentials to the provision service you can pass it in as an environment variable or in a config file as Mainflux user and password or as API token (that can be issued on `/users` or `/keys` endpoint of [authn][authn]. + +Additionally, users or API token can be passed in Authorization header, this authentication takes precedence over others. + +* `username`, `password` - (`MF_PROVISION_USER`, `MF_PROVISION_PASSWORD` in [.env][env], `mf_user`, `mf_pass` in [config.toml][conftoml] +* API Key - (`MF_PROVISION_API_KEY` in [.env][env] or [config.toml][conftoml] +* `Authorization: Token|ApiKey` - request authorization header containing either users token or API key. Check [authn][authn]. + +## Running +Provision service can be run as a standalone or in docker composition as addon to the core docker composition. + +Standalone: +```bash +MF_PROVISION_BS_SVC_URL=http://localhost:8202/things \ +MF_PROVISION_THINGS_LOCATION=http://localhost:8182 \ +MF_PROVISION_USERS_LOCATION=http://localhost:8180 \ +MF_PROVISION_CONFIG_FILE=docker/addons/provision/configs/config.toml \ +build/mainflux-provision +``` + +Docker composition: +```bash +docker-compose -f docker/addons/provision/docker-compose.yml up +``` + +## Provision + +For the case that credentials or API token is passed in configuration file or environment variables, call to `/mapping` endpoint doesn't require `Authentication` header: +```bash +curl -s -S -X POST http://localhost:8888/mapping -H 'Content-Type: application/json' -d '{"external_id": "33:52:77:99:43", "external_key": "223334fw2"}' +``` + +In the case that provision service is not deployed with credentials or API key or you want to use user other than one being set in environment (or config file): +```bash +curl -s -S -X POST http://localhost:8091/mapping -H "Authorization: " -H 'Content-Type: application/json' -d '{"external_id": "", "external_key": ""}' +``` + +Or if you want to specify a name for thing different than in `config.toml` you can specify post data as: + +```json +{"name": "", "external_id": "", "external_key": ""} +``` + +Response contains created things, channels and certificates if any: +```json +{ + "things": [ + { + "id": "c22b0c0f-8c03-40da-a06b-37ed3a72c8d1", + "name": "thing", + "key": "007cce56-e0eb-40d6-b2b9-ed348a97d1eb", + "metadata": { + "external_id": "33:52:79:C3:43" + } + } + ], + "channels": [ + { + "id": "064c680e-181b-4b58-975e-6983313a5170", + "name": "control-channel", + "metadata": { + "type": "control" + } + }, + { + "id": "579da92d-6078-4801-a18a-dd1cfa2aa44f", + "name": "data-channel", + "metadata": { + "type": "data" + } + } + ], + "whitelisted": { + "c22b0c0f-8c03-40da-a06b-37ed3a72c8d1": true + } +} +``` + +[mainflux]: https://github.com/mainflux/mainflux +[bootstrap]: https://github.com/mainflux/mainflux/tree/master/bootstrap +[export]: https://github.com/mainflux/export +[agent]: https://github.com/mainflux/agent +[mfxui]: https://github.com/mainflux/mainflux/ui +[config]: https://github.com/mainflux/mainflux/tree/master/provision#configuration +[env]: https://github.com/mainflux/mainflux/blob/master/.env +[conftoml]: https://github.com/mainflux/mainflux/blob/master/docker/addons/provision/configs/config.toml +[authn]: https://github.com/mainflux/mainflux/blob/master/authn/README.md +[exp]: https://github.com/mainflux/export +[cli]: https://github.com/mainflux/mainflux/tree/master/cli \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 3754a0a6..a8d430aa 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -27,7 +27,9 @@ pages: - License: LICENSE.txt - Architecture: architecture.md - Getting Started: getting-started.md - - Provisioning: provisioning.md + - Provision: + - Platform management: platform-management.md + - Provision Service: provision.md - Messaging: messaging.md - Storage: storage.md - LoRa: lora.md