From cec19c68023f4521bde02d7645fdb2ebfc855d8d Mon Sep 17 00:00:00 2001 From: edgex-jenkins Date: Fri, 20 Dec 2024 01:10:30 +0000 Subject: [PATCH] ci: Merge pull request #1407 from iain-anderson/auto-opstate-branch Signed-off-by: edgex-jenkins --- .../device/Configuration/index.html | 10 + 4.0/search/search_index.json | 2 +- 4.0/sitemap.xml | 646 +++++++++--------- 4.0/sitemap.xml.gz | Bin 3236 -> 3237 bytes 4 files changed, 334 insertions(+), 324 deletions(-) diff --git a/4.0/microservices/device/Configuration/index.html b/4.0/microservices/device/Configuration/index.html index 02bb89650..d3a255436 100644 --- a/4.0/microservices/device/Configuration/index.html +++ b/4.0/microservices/device/Configuration/index.html @@ -3027,6 +3027,16 @@

Device Service - Configuration

Size of the buffer for async readings +AllowedFails +0 +If set, number of consecutive failures to access a device after which the service will set the device's OperationalState to DOWN + + +DeviceDownTimeout +0 +If set, an interval in seconds after which a device which has been automatically set DOWN will be re-tried to see if it is accessible again + + Discovery/Enabled false Controls whether device discovery is enabled diff --git a/4.0/search/search_index.json b/4.0/search/search_index.json index c23302836..ae32f7860 100644 --- a/4.0/search/search_index.json +++ b/4.0/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Overview","text":"

EdgeX Foundry is a vendor-neutral open source project hosted by The Linux Foundation. EdgeX Foundry builds a common open framework for IoT edge computing. At the heart of the project is an interoperability framework hosted within a full hardware- and OS-agnostic reference software platform to enable an ecosystem of plug-and-play components that unifies the marketplace and accelerates the deployment of IoT solutions.

Docker Quick Start

Jump in to EdgeX Foundry by running locally with Docker containers.

Build & Run Natively

Build EdgeX and run it natively on your OS.

Build a Device Service

Build a custom device service to connect to your sensor or device.

Build an Application Service

Build or configure a new application service to get data to the cloud, database, enterprise application or other external system.

Running in Hybrid Mode

How to run a service you are working on natively and then run the rest of EdgeX with Docker containers

"},{"location":"V3TopLevelMigration/","title":"V3 Migration Guide","text":"

EdgeX 3.0

Many backward breaking changes occurred in the EdgeX 3.0 (Minnesota) release which may require some migration depending on your use case.

This section describes how to migrate from V2 to V3 at a high level and refers the reader to the appropriate detail documents. The areas to consider for migrating are:

"},{"location":"V3TopLevelMigration/#customized-configuration","title":"Customized Configuration","text":"

Service configuration is one of the big changes for EdgeX V3

"},{"location":"V3TopLevelMigration/#configuration-provider","title":"Configuration Provider","text":"

If you have customized any EdgeX service's configuration (core, support, device, etc.) via the Configuration Provider (Consul), those customization will need to be re-applied to those services' configuration or the common configuration in the Configuration Provider once the V3 versions have started and pushed their configuration into the Configuration Provider. The V3 services now use v3 in the Configuration Provider path rather than 2.0 . The folder structure in the Configuration Provider has been flattened so all services are at the same level. See the Configuration File section below for details on migrating configuration.

Example Configuration Provider paths for V3

.../kv/edgex/v3/core-common-config-bootstrapper\n.../kv/edgex/v3/core-data/\n.../kv/edgex/v3/device-virtual/\n.../kv/edgex/v3/app-rules-engine/\n

The same applies for custom device and application service once they have been migrated following the guides referenced in the Custom Device Service and Custom Applications Service sections below.

Warning

If the Configuration Provider data is not cleared prior to running the V3 services, the V2 configuration will remain and be taking up useful memory. The configuration data in the Configuration Provider can be cleared by deleting the .../edgex/ node with the curl command below prior to starting EdgeX 3.0. 

curl --request DELETE http://localhost:8500/v1/kv/edgex?recurse=true`\n

"},{"location":"V3TopLevelMigration/#configuration-file","title":"Configuration File","text":"

If you have customized the service configuration files for any EdgeX service (core, support, device, etc.) that configuration will need to be migrated.

The biggest two changes to the service configuration files are:

  1. File format has changed to YAML
  2. Settings that are common have been removed from each service's local private configuration file

See V3 Migration of Common Configuration for the details on migrating configuration common to all EdgeX services.

The tool here can be used to convert your customized service configuration file from TOML to YAML. This should be done once all the common configuration has been removed.

The following are where you can find the configuration migration specifics for individual EdgeX services

"},{"location":"V3TopLevelMigration/#customized-environment-overrides","title":"Customized Environment Overrides","text":"

If you have custom environment overrides for configuration impacted by the V3 changes you will also need to migrate your overrides to use the new name or value depending on what has changed. Refer to the links above and/or below for details for migration of common and/or the service specific configuration to determine if your overrides require migrating.

Note

When using the Configuration Provider, the environment overrides for common configuration are applied to the core-common-config-bootstrapper service. They no longer work when applied to the individual services as the common configuration setting no longer exist in the private configuration.

"},{"location":"V3TopLevelMigration/#custom-compose-file","title":"Custom Compose File","text":"

The compose files for V3 have many changes from their V2 counter parts. If you have customized a V2 compose file to add additional services and/or add or modify configuration overrides, it is highly recommended that you start with the appropriate V3 compose file and re-add your customizations. It is very likely that the sections for your additional services will need to be migrated to have the proper environment overrides. Best approach is to use one of the V3 service sections that closest matches your service as a template.

The latest V3 compose files can be found here: Compose Files

"},{"location":"V3TopLevelMigration/#compose-builder","title":"Compose Builder","text":"

If the additional service(s) in your custom compose file are EdgeX released device or app services, it is highly recommended that you use the Compose Builder to regenerate your custom compose file.

The latest V3 Compose Builder can be found here: Compose Builder Readme

"},{"location":"V3TopLevelMigration/#command-line-options","title":"Command Line Options","text":"

The following command-line options and corresponding environment variables have be renamed for consistency

If your solution uses any of the renamed options or environment variables you will need to make the appropriate changes to use the new names.

See Command Line Options page for more details on the above options and the Command Line Overrides section for more details on the above environment variables

"},{"location":"V3TopLevelMigration/#database","title":"Database","text":"

There currently is no migration path for the data stored in the database. If possible, the database should be cleared prior to starting V3 EdgeX. This will allow the database to be V3 compliant from the start. See Clearing Redis Database section below for details on how to clear the Redis database.

The following sections describe what you need to be aware for the different services that create data in the database.

"},{"location":"V3TopLevelMigration/#core-data","title":"Core Data","text":"

The Event/Reading data stored by Core Data is considered transient and of little value once it has become old. The V3 versions of these data collections have minimal changes from their V2 counter parts.

"},{"location":"V3TopLevelMigration/#api-change","title":"API Change","text":"

See Core Data API Reference for complete details.

"},{"location":"V3TopLevelMigration/#reading","title":"Reading","text":""},{"location":"V3TopLevelMigration/#event","title":"Event","text":"

The field that has changed in V3 is the apiVersion which is now set to v3.

"},{"location":"V3TopLevelMigration/#core-metadata","title":"Core Metadata","text":"

Most of the data stored by Core Metadata will be recreated when the V3 versions of the Device Services start-up. The statically declared devices will automatically be created and device discovery will find and add existing devices. Any device profiles, devices, provision watchers created manually via the V2 REST APIs will have to be recreated using the V3 REST API. Any manually-applied AdministrativeState settings will also need to be re-applied.

"},{"location":"V3TopLevelMigration/#api-change_1","title":"API Change","text":"

See Core Metadata API Reference for complete details.

"},{"location":"V3TopLevelMigration/#core-command","title":"Core Command","text":""},{"location":"V3TopLevelMigration/#api-change_2","title":"API Change","text":"

See Core Command API Reference for complete details.

"},{"location":"V3TopLevelMigration/#support-notifications","title":"Support Notifications","text":"

Any Subscriptions created via the V2 REST API will have to be recreated using the V3 REST API. The Notification and Transmissioncollections will be empty until new notifications are sent using EdgeX V3

"},{"location":"V3TopLevelMigration/#support-scheduler","title":"Support Scheduler","text":""},{"location":"V3TopLevelMigration/#api-change_3","title":"API Change","text":"

The statically declared Interval and IntervalAction will be created automatically. Any Interval and/or IntervalAction created via the V2 REST API will have to be recreated using the V3 REST API. If you have created a custom configuration with additional statically declared Intervals and IntervalActions see the Configuration File section under Customized Configuration below.

"},{"location":"V3TopLevelMigration/#application-services","title":"Application Services","text":"

Application services use the database only when the Store and Forward capability is enabled. If you do not use this capability you can skip this section. This data collection only has data when that data could not be exported. It is recommended not to upgrade to V3 while the Store and Forward data collection is not empty or you are certain the data is no longer needed. You can determine if the Store and Forward data collection is empty by setting the Application Service's log level to DEBUG and look for the following message which is logged every RetryInterval:

Example

msg=\" 0 stored data items found for retrying\"\n

Note

The RetryInterval is in the app-services section of common configuration. Changing it there will apply to all Application Services that have the Store and Forward capability enabled.

"},{"location":"V3TopLevelMigration/#clearing-redis-database","title":"Clearing Redis Database","text":""},{"location":"V3TopLevelMigration/#docker","title":"Docker","text":"

When running EdgeX in Docker the simplest way to clear the database is to remove the db-data volume after stopping the V2 EdgeX services. 

docker compose -f <compose-file> down\ndocker volume rm $(docker volume ls -q | grep db-data)\n

Now when the V3 EdgeX services are started the database will be cleared of the old v2 data.

"},{"location":"V3TopLevelMigration/#local","title":"Local","text":"

If you are running EdgeX locally, i.e. not in Docker and in non-secure mode you can use the Redis CLI to clear the database. The CLI would have been installed when you installed Redis locally. Run the following command to clear the database:

redis-cli FLUSHDB\n

This will not work if running EdgeX in running in secure mode since you will not have the random generated Redis password unless you created an Admin password when you installed Redis.

"},{"location":"V3TopLevelMigration/#custom-device-service","title":"Custom Device Service","text":"

If you have custom Device Services they will need to be migrated to the V3 version of the Device SDK. See Device Service V3 Migration Guide for complete details.

"},{"location":"V3TopLevelMigration/#custom-device-profile","title":"Custom Device Profile","text":"

If you have custom V2 Device Profile(s) for one of the EdgeX Device Services they will need to be migrated to the V3 version of Device Profiles. See Device Service V3 Migration Guide for complete details.

"},{"location":"V3TopLevelMigration/#custom-pre-defined-device","title":"Custom Pre-Defined Device","text":"

If you have custom V2 Pre-Defined Device(s) for one of the EdgeX Device Services they will need to be migrated to the V3 version of Pre-Defined Devices. See Device Service V3 Migration Guide for complete details.

"},{"location":"V3TopLevelMigration/#custom-applications-service","title":"Custom Applications Service","text":"

If you have custom Application Services they will need to be migrated to the V3 version of the App Functions SDK. See Application Services V3 Migration Guide for complete details.

"},{"location":"V3TopLevelMigration/#security","title":"Security","text":"

If you have an add-on services running in secure mode you will need to use the new names of the environment variables in EdgeX V3. See Security Services V3 Migration Guide for more details.

"},{"location":"V3TopLevelMigration/#api-gateway-configuration","title":"API Gateway configuration","text":"

The API gateway has changed in EdgeX V3. See Security Services V3 Migration Guide for more details.

"},{"location":"V3TopLevelMigration/#authenticated-rest-apis","title":"Authenticated REST APIs","text":"

When security is enable, all V3 EdgeX services REST APIs require a JWT authorization token. See Security Services V3 Migration Guide for more details.

"},{"location":"V3TopLevelMigration/#ekuiper","title":"eKuiper","text":""},{"location":"V3TopLevelMigration/#rules","title":"Rules","text":""},{"location":"V3TopLevelMigration/#rest-action","title":"Rest Action","text":""},{"location":"V3TopLevelMigration/#none-secure-mode","title":"None Secure Mode","text":"

If running EdgeX in none secure mode and you have rules with rest action that reference an EdgeX service the endpoint API version will need to be changed from v2 to V3

Example migration of rest action with EdgeX endpoint

V2: 

\"actions\": [\n{\n\"rest\": {\n\"url\": \"http://edgex-core-command:59882/api/v2/device/name/Random-Integer-Device/Int64\",       ...\n}\n}\n]\n

\u200b V3: 

\"actions\": [\n{\n\"rest\": {\n\"url\": \"http://edgex-core-command:59882/api/v3/device/name/Random-Integer-Device/Int64\",       ...\n}\n}\n]\n

"},{"location":"V3TopLevelMigration/#secure-mode","title":"Secure Mode","text":"

If running EdgeX in secure mode and you have rules with rest action that reference an EdgeX Core Command you will need to convert the rule to use Command via External MQTT. See eKuiper documentation here for more details. This is due to the new microservice authorization on all EdgeX services' endpoints requiring a JWT token which eKuiper doesn't have.

Note

This approach requires an external MQTT broker to send the command requests. The default EdgeX compose files do not include a MQTT Broker. This broker is supposed to be external to EdgeX.

"},{"location":"about/","title":"About","text":"

EdgeX Foundry is an open source, vendor neutral, flexible, interoperable, software platform at the edge of the network, that interacts with the physical world of devices, sensors, actuators, and other IoT objects. In simple terms, EdgeX is edge middleware - serving between physical sensing and actuating \"things\" and our information technology (IT) systems.

The EdgeX platform enables and encourages the rapidly growing community of IoT solution providers to work together in an ecosystem of interoperable components to reduce uncertainty, accelerate time to market, and facilitate scale.

By bringing this much-needed interoperability, EdgeX makes it easier to monitor physical world items, send instructions to them, collect data from them, move the data across the fog up to the cloud where it may be stored, aggregated, analyzed, and turned into information, actuated, and acted upon. So EdgeX enables data to travel northwards towards the cloud or enterprise and back to devices, sensors, and actuators.

The initiative is aligned around a common goal: the simplification and standardization of the foundation for tiered edge computing architectures in the IoT market while still enabling the ecosystem to provide significant value-added differentiation.

If you don't need further description and want to immediately use EdgeX Foundry use this link: Getting Started Guide

"},{"location":"about/#edgex-foundry-use-cases","title":"EdgeX Foundry Use Cases","text":"

Originally built to support industrial IoT needs, EdgeX today is used in a variety of use cases to include:

"},{"location":"about/#edgex-foundry-architectural-tenets","title":"EdgeX Foundry Architectural Tenets","text":"

EdgeX Foundry was conceived with the following tenets guiding the overall architecture:

"},{"location":"about/#deployments","title":"Deployments","text":"

EdgeX was originally built by Dell to run on its IoT gateways. While EdgeX can and does run on gateways, its platform agnostic nature and micro service architecture enables tiered distributed deployments. In other words, a single instance of EdgeX\u2019s micro services can be distributed across several host platforms. The host platform for one or many EdgeX micro services is called a node. This allows EdgeX to leverage compute, storage, and network resources wherever they live on the edge.

Its loosely-coupled architecture enables distribution across nodes to enable tiered edge computing. For example, thing communicating services could run on a programmable logic controller (PLC), a gateway, or be embedded in smarter sensors while other EdgeX services are deployed on networked servers. The scope of a deployment could therefore include embedded sensors, controllers, edge gateways, servers and cloud systems.

EdgeX micro services can be deployed across an array of compute nodes to maximize resources while at the same time position more processing intelligence closer to the physical edge. The number and the function of particular micro services deployed on a given node depends on the use case and capability of the hardware and infrastructure.

"},{"location":"about/#apache-2-license","title":"Apache 2 License","text":"

EdgeX is distributed under Apache 2 License backed by the Apache Foundation. Apache 2 licensing is very friendly (\u201cpermissive\u201d) to open and commercial interests. It allows users to use the software for any purpose. It allows users to distribute, modify or even fork the code base without seeking permission from the founding project. It allows users to change or extend the code base without having to contribute back to the founding project. It even allows users to build commercial products without concerns for profit sharing or royalties to go back to the Linux Foundation or open source project organization.

"},{"location":"about/#edgex-foundry-service-layers","title":"EdgeX Foundry Service Layers","text":"

EdgeX Foundry is a collection of open source micro services. These micro services are organized into 4 service layers, and 2 underlying augmenting system services. The Service Layers traverse from the edge of the physical realm (from the Device Services Layer), to the edge of the information realm (that of the Application Services Layer), with the Core and Supporting Services Layers at the center.

The 4 Service Layers of EdgeX Foundry are as follows:

The 2 underlying System Services of EdgeX Foundry are as follows:

"},{"location":"about/#core-services-layer","title":"Core Services Layer","text":"

Core services provide the intermediary between the north and south sides of EdgeX. As the name of these services implies, they are \u201ccore\u201d to EdgeX functionality. Core services is where most of the innate knowledge of what \u201cthings\u201d are connected, what data is flowing through, and how EdgeX is configured resides in an EdgeX instance. Core consists of the following micro services:

Core services provide intermediary communications between the things and the IT systems.

"},{"location":"about/#supporting-services-layer","title":"Supporting Services Layer","text":"

The supporting services encompass a wide range of micro services to include edge analytics (also known as local analytics). Normal software application duties such as scheduler, and data clean up (also known as scrubbing in EdgeX) are performed by micro services in the supporting services layer.

These services often require some amount of core services in order to function. In all cases, supporting service can be considered optional \u2013 that is they can be left out of an EdgeX deployment depending on use case needs and system resources.

Supporting services include:

"},{"location":"about/#application-services-layer","title":"Application Services Layer","text":"

Application services are the means to extract, process/transform and send sensed data from EdgeX to an endpoint or process of your choice. EdgeX today offers application service examples to send data to many of the major cloud providers (Amazon IoT Hub, Google IoT Core, Azure IoT Hub, IBM Watson IoT\u2026), to MQTT(s) topics, and HTTP(s) REST endpoints.

Application services are based on the idea of a \"functions pipeline\". A functions pipeline is a collection of functions that process messages (in this case EdgeX event messages) in the order specified. The first function in a pipeline is a trigger. A trigger begins the functions pipeline execution. A trigger, for example, is something like a message landing in a message queue. Each function then acts on the message. Common functions include filtering, transformation (i.e. to XML or JSON), compression, and encryption functions. The function pipeline ends when the message has gone through all the functions and is set to a sink. Putting the resulting message into an MQTT topic to be sent to Azure or AWS is an example of a sink completing an application service.

"},{"location":"about/#device-services-layer","title":"Device Services Layer","text":"

Device services connect \u201cthings\u201d \u2013 that is sensors and devices \u2013 into the rest of EdgeX.

Device services are the edge connectors interacting with the \"things\" that include, but are not limited to: alarm systems, heating and air conditioning systems in homes and office buildings, lights, machines in any industry, irrigation systems, drones, currently automated transit such as some rail systems, currently automated factories, and appliances in your home. In the future, this may include driverless cars and trucks, traffic signals, fully automated fast food facilities, fully automated self-serve grocery stores, devices taking medical readings from patients, etc.

Device services may service one or a number of things or devices (sensor, actuator, etc.) at one time. A device that a device service manages, could be something other than a simple, single, physical device. The device could be another gateway (and all of that gateway's devices), a device manager, a device aggregator that acts as a device, or collection of devices, to EdgeX Foundry.

The device service communicates with the devices, sensors, actuators, and other IoT objects through protocols native to each device object. The device service converts the data produced and communicated by the IoT object into a common EdgeX Foundry data structure, and sends that converted data into the core services layer, and to other micro services in other layers of EdgeX Foundry.

EdgeX comes with a number of device services speaking many common IoT protocols such as Modbus, BACnet, MQTT, etc.

"},{"location":"about/#system-services-layer","title":"System Services Layer","text":"

Security Infrastructure

Security elements of EdgeX Foundry protect the data and control of devices, sensors, and other IoT objects managed by EdgeX Foundry. Based on the fact that EdgeX is a \"vendor-neutral open source software platform at the edge of the network\", the EdgeX security features are also built on a foundation of open interfaces and pluggable, replaceable modules.

There are two major EdgeX security components.

"},{"location":"about/#software-development-kits-sdks","title":"Software Development Kits (SDKs)","text":"

Two types of SDKs are provided by EdgeX to assist in creating north and south side services \u2013 specifically to create application services and device services. SDKs for both the north and south side services make connecting new things or new cloud/enterprise systems easier by providing developers all the scaffolding code that takes care of the basic operations of the service. Thereby allowing developers to focus on specifics of their connectivity to the south or north side object without worrying about all the raw plumbing of a micro service.

SDKs are language specific; meaning an SDK is written to create services in a particular programming language. Today, EdgeX offers the following SDKs:

"},{"location":"about/#how-edgex-works","title":"How EdgeX Works","text":""},{"location":"about/#sensor-data-collection","title":"Sensor Data Collection","text":"

EdgeX\u2019s primary job is to collect data from sensors and devices and make that data available to north side applications and systems. Data is collected from a sensor by a device service that speaks the protocol of that device. Example: a Modbus device service would communicate in Modbus to get a pressure reading from a Modbus pump. The device service translates the sensor data into an EdgeX event object. The device service can then either:

  1. put the event object on a message bus (which may be implemented via MQTT). Subscribers to the event message on the message bus can be application services or core data or both (see step 1.1 below).

  2. send the event object to the core data service via REST communications (see step 1.2).

When core data receives the event (either via message bus or REST), it persists the sensor data in the local edge database. EdgeX uses PostgreSQL as our persistence store. There is an abstraction in place to allow you to use another database (which has allowed other databases to be used in the past). Persistence is not required and can be turned off. Data is persisted in EdgeX at the edge for two basics reasons:

When core data receives event objects from the device service via REST, it will put sensor data events on a message topic destined for application services. MQTT is used as the messaging infrastructure by default (step 2). NATS (opt-in during build) can also be used as the messaging infrastructure between core data and the application services.

The application service transforms the data as needed and pushes the data to an endpoint. It can also filter, enrich, compress, encrypt or perform other functions on the event before sending it to the endpoint (step 3). The endpoint could be an HTTP/S endpoint, an MQTT topic, a cloud system (cloud topic), etc.

"},{"location":"about/#edge-analytics-and-actuation","title":"Edge Analytics and Actuation","text":"

In edge computing, simply collecting sensor data is only part of the job of an edge platform like EdgeX. Another important job of an edge platform is to be able to:

Why edge analytics? Local analytics are important for two reasons:

Local analytics allows systems to operate independently, at least for some stretches of time. For example: a shipping container\u2019s cooling system must be able to make decisions locally without the benefit of Internet connectivity for long periods of time when the ship is at sea. Local analytics also allow a system to act quickly in a low latent fashion when critical to system operations. As an extreme case, imagine that your car\u2019s airbag fired on the basis of data being sent to the cloud and analyzed for collisions. Your car has local analytics to prevent such a potentially slow and error prone delivery of the safety actuation in your automobile.

EdgeX is built to act locally on data it collects from the edge. In other words, events are processed by local analytics and can be used to trigger action back down on a sensor/device.

Just as application services prepare data for consumption by north side cloud systems or applications, application services can process and get EdgeX events (and the sensor data they contain) to any analytics package (see step 4). By default, EdgeX ships with a simple rules engine (the default EdgeX rules engine is eKuiper \u2013 an open source rules engine and now a sister project in LF Edge). Your own analytics package (or ML agent) could replace or augment the local rules engine.

The analytic package can explore the sensor event data and make a decision to trigger actuation of a device. For example, it could check that the pressure reading of an engine is greater than 60 PSI. When such a rule is determined to be true, the analytic package calls on the core command service to trigger some action, like \u201copen a valve\u201d on some controllable device (see step 5).

The core command service gets the actuation request and determines which device it needs to act on with the request; then calling on the owning device service to do the actuation (see step 6). Core command allows developers to put additional security measures or checks in place before actuating.

The device service receives the request for actuation, translates that into a protocol specific request and forwards the request to the desired device (see step 7).

"},{"location":"about/#project-release-cadence","title":"Project Release Cadence","text":"

Typically, EdgeX releases twice a year; once in the spring and once in the fall. Bug fix releases may occur more often. Each EdgeX release has a code name. The code name follows an alphabetic pattern similar to Android (code names sequentially follow the alphabet).

The code name of each release is named after some geographical location in the world. The honor of naming an EdgeX release is given to a community member deemed to have contributed significantly to the project. A release also has a version number. The release version follows sematic versioning to indicate the release is major or minor in scope. Major releases typically contain significant new features and functionality and are not always backward compatible with prior releases. Minor releases are backward compatible and usually contain bug fixes and fewer new features. See the project Wiki for more information on releases, versions and patches. | Release | Schedule | Version | | :------------- | :----------: | -----------: | |Barcelona |Oct 2017 | 0.5.0 | |California |Jun 2017 | 0.6.0 | |Delhi |Oct 2018 | 0.7.0 | |Edinburgh |Jul 2019 | 1.0.0 | |Fuji |Nov 2019 | 1.1.0 | |Geneva |May 2020 | 1.2.0 | |Hanoi |November 2020 | 1.3.0 | |Ireland |Spring 2021 | 2.0.0 | |Jakarta |Fall 2021 | 2.1.0 | |Kamukura |Spring 2022| TBD | |Levski| Fall 2022| TBD|

Note: minor releases of the Device Services and Application Services (along with their associated SDKs) can be release independently. Graphical User Interface, the command line interface (CLI) and other tools can be released independently.

EdgeX community members convene in a meeting right at the time of a release to plan the next release and roadmap future releases.

See the Project Wiki for more detailed information on releases and roadmap.

"},{"location":"about/#edgex-history-and-naming","title":"EdgeX History and Naming","text":"

EdgeX Foundry began as a project chartered by Dell IoT Marketing and developed by the Dell Client Office of the CTO as an incubation project called Project Fuse in July 2015. It was initially created to run as the IoT software application on Dell\u2019s introductory line of IoT gateways. Dell entered the project into open source through the Linux Foundation on April 24, 2017. EdgeX was formally announced and demonstrated at Hanover Messe 2017. Hanover Messe is one of the world's largest industrial trade fairs. At the fair, the Linux Foundation also announced the association of 50 founding member organizations \u2013 the EdgeX ecosystem \u2013 to help further the project and the goals of creating a universal edge platform.

The name \u2018foundry\u2019 was used to draw parallels to Cloud Foundry. EdgeX Foundry is meant to be a foundry for solutions at the edge just like Cloud Foundry is a foundry for solutions in the cloud. Cloud Foundry was originated by VMWare (Dell Technologies is a major shareholder of VMWare - recall that Dell Technologies was the original creator of EdgeX). The \u2018X\u2019 in EdgeX represents the transformational aspects of the platform and allows the project name to be trademarked and to be used in efforts such as certification and certification marks.

The EdgeX Foundry Logo represents the nature of its role as transformation engine between the physical OT world and the digital IT world.

The EdgeX community selected the octopus as the mascot or \u201cspirit animal\u201d of the project at its inception. Its eight arms and the suckers on the arms represent the sensors. The sensors bring the data into the octopus. Actually, the octopus has nine brains in a way. It has millions of neurons running down each arm; functioning as mini-brains in each of those arms. The arms of the octopus serve as \u201clocal analytics\u201d like that offered by EdgeX. The mascot is affectionately called \u201cEdgey\u201d by the community.

"},{"location":"api/Ch-APIIntroduction/","title":"Introduction","text":"

Each of the EdgeX services (core, supporting, management, device and application) implement a RESTful API. This section provides details about each service's API. You will see there is a common set of API's that all services implement, which are:

Each Edgex Service's RESTful API is documented via Swagger. A link is provided to the swagger document in the service specific documentation.

Also included in this API Reference are a couple 3rd party services (Configuration/Registry and Rules Engine). These services do not implement the above common APIs and don't not have swagger documentation. Links are provided to their appropriate documentation.

See the left side navigation for complete list of services to access their API Reference.

"},{"location":"api/applications/Ch-APIRulesEngine/","title":"Rules Engine","text":"

EdgeX Foundry Rules Engine Microservice receives data from the instance of App Service Configurable running the rules-engine profile (aka app-rules-engine) via the EdgeX MessageBus. EdgeX uses eKuiper for the rules engine, which is a separate LF Edge project. See the eKuiper Website for more details on this rules engine.

eKuiper's documentation

"},{"location":"api/core/Ch-APICoreCommand/","title":"Core Command","text":"

EdgeX Foundry's Command microservice is a conduit for other services to trigger action on devices and sensors through their managing Device Services. See Core Command for more details about this service.

The service provides an API to get the list of commands that can be issued for all devices or a single device. Commands are divided into two groups for each device:

"},{"location":"api/core/Ch-APICoreCommand/#swagger","title":"Swagger","text":""},{"location":"api/core/Ch-APICoreConfigurationAndRegistry/","title":"Configuration and Registry","text":"

EdgeX uses a new microservice, Core Keeper, for Configuration Provider and Service Registry functions. Core Keeper integrates PostgreSQL for data persistence and leverages EdgeX modules go-mod-configuration and go-mod-registry to implement these services within the EdgeX architecture.

"},{"location":"api/core/Ch-APICoreConfigurationAndRegistry/#configuration-management","title":"Configuration Management","text":"

For current Configuration Management, refer to the Core Keeper API documentation.

"},{"location":"api/core/Ch-APICoreConfigurationAndRegistry/#service-registry","title":"Service Registry","text":"

For detailed API documentation on Core Keeper's service registry, refer to the Core Keeper API documentation.

Service Registration

When each microservice starts up, it connects to Core Keeper to register its endpoint details, including microservice ID, address, port, and health-check methods. Other microservices can then discover its URL through Core Keeper, which also monitors health status. API details for registration are provided in the Core Keeper API documentation.

Service Deregistration

Before shutting down, microservices must deregister from Core Keeper. The deregistration API is described in the Core Keeper API documentation.

Service Discovery

The Service Discovery feature enables client microservices to query endpoint information for specific microservices by their ID or to list all available services registered in Core Keeper. Refer to the Core Keeper API documentation.

Health Checking

Health checking ensures only healthy services are used. Core Keeper offers HTTP + Interval as the health checking method, see the Core Keeper API documentation.

Health checks should be established during service registration, as detailed in the Service Registration section.

"},{"location":"api/core/Ch-APICoreKeeper/","title":"Core Keeper","text":"

EdgeX Foundry's Core Keeper microservice is used for Configuration and Registry. See Configuration and Registry Providers for more details about Configuration and Registry.

"},{"location":"api/core/Ch-APICoreKeeper/#swagger","title":"Swagger","text":""},{"location":"api/core/Ch-APICoreMetadata/","title":"Core Metadata","text":"

The Core Metadata microservice includes the device/sensor metadata database and APIs to expose this database to other services. In particular, the device provisioning service deposits and manages device metadata through this service's API. See Core Metadata for more details about this service.

"},{"location":"api/core/Ch-APICoreMetadata/#swagger","title":"Swagger","text":""},{"location":"api/core/Ch-APICoredata/","title":"Core Data","text":"

EdgeX Foundry Core Data microservice communicates with the database to store Events/Readings collected from devices and sensors. It has APIs to expose this database to other services to allow these services to Add, Query and Delete Events/Readings. See Core Data for more details about this service.

"},{"location":"api/core/Ch-APICoredata/#swagger","title":"Swagger","text":""},{"location":"api/devices/Ch-APIDeviceSDK/","title":"Device Services","text":"

The EdgeX Foundry Device Service Software Development Kit (SDK) takes the Developer through the step-by-step process to create an EdgeX Foundry Device Service microservice. See Device Service SDK for more details on this SDK.

The Device Service SDK provides a RESTful API that all Device Services inherit from the SDK.

"},{"location":"api/devices/Ch-APIDeviceSDK/#swagger","title":"Swagger","text":""},{"location":"api/support/Ch-APISupportNotifications/","title":"Support Notifications","text":"

When a person or a system needs to be informed of something discovered on the node by another microservice on the node, EdgeX Foundry's Support Notifications microservice delivers that information. Examples of Alerts and Notifications that other services might need to broadcast include sensor data detected outside of certain parameters, usually detected by a Rules Engine service, or a system or service malfunction usually detected by system management services. See Support Notifications for more details about this service.

"},{"location":"api/support/Ch-APISupportNotifications/#swagger","title":"Swagger","text":""},{"location":"api/support/Ch-APISupportScheduler/","title":"Support Scheduler","text":"

EdgeX Foundry's Support Scheduler microservice to schedule actions to occur on specific intervals or crontab definition. See Support Scheduler (TBD) for more details about this service.

"},{"location":"api/support/Ch-APISupportScheduler/#swagger","title":"Swagger","text":""},{"location":"design/Process/","title":"Use Cases and Design Process","text":"

This document describes the EdgeX use case driven requirements engineering and design process.

Approved by consent of the TSC on 2022-07-13

Supersedes the processes documented on the EdgeX Wiki

"},{"location":"design/Process/#use-case-driven-approach-to-requirements-and-design","title":"Use Case Driven Approach to Requirements and Design","text":"

Designing an architecture is a very time consuming task. It is best to start that with a solid foundation. The obvious goal is to design an architecture that satisfies the functional requirements, while being secure, flexible, and robust. Requirements are very important factors when designing a system. They should be derived from established, validated, and most importantly, written use cases. To avoid feature creep, the architecture should focus on requirements that are backed by multiple use cases and in the meantime try to remain extensible.

The following figure outlines the EdgeX process around use cases, requirements capture, and architectural design.

"},{"location":"design/Process/#use-cases-and-requirements","title":"Use Cases and Requirements","text":"

In any software system, new needs of the software are encountered on a regular basis. Any need that is more than a request to fix a bug or make a minor addition/change to the software should be added as feature requests (on Github) and supported by written use cases. The use cases should be documented in an EdgeX Use Case Record (UCR). UCRs must be reviewed by domain experts and approved by the TSC per the process documented here.

"},{"location":"design/Process/#ucr-template","title":"UCR template","text":"

UCRs should be submitted as pull requests against the UCR area of edgex-docs. Use the current UCR template to help create the UCR document.

"},{"location":"design/Process/#ucr-review-and-approval-process","title":"UCR Review and Approval Process","text":"

The community can submit UCR. The use cases describe the use case, target users, data, hardware, privacy and security considerations. Each use case should also include a list of functional requirements, the list of existing tools (that satisfy those requirements) and gaps. Use cases and requirements may freely overlap. Submissions get peer reviewed by domain experts and TSC. The TSC approves UCR and allows design work to be conducted based on the requirements. They can be updated to address shortcomings and technological advancements. Once a stable implementation is available addressing all the requirements, the record gets classified as \"supported\".

"},{"location":"design/Process/#designs","title":"Designs","text":"

Issues and new requirements lead to design decisions. Design decisions are also made on a regular, if not daily, basis. Some of these decisions are big and impactful to all parts of the system. Other decisions are less significant but still important for everyone to know and understand.

EdgeX has two places to record design decisions.

Note: ADRs should also be documented on the project board with a link to the ADR in edgex-docs in the project board card.

"},{"location":"design/Process/#when-to-use-an-adr","title":"When to use an ADR","text":"

\"Significant architectural decisions\" are deemed those that:

Impact more than one EdgeX service and often impact the entire system (such as the definition of a data transfer object used through the system, of a feature that must be supported by all services).\nRequire a lot of manpower (more than two people working over the course of a release or more) to implement the feature outlined in the ADR.\nRequires implementation to be accomplished over multiple releases (either due to the complexity of the feature or dependencies).\n

ADRs must be proceeded by one or more approved UCRs in order to be approved by the TSC - allowing for the design to be implemented in the EdgeX software.

"},{"location":"design/Process/#adr-template","title":"ADR template","text":"

ADRs should be submitted as pull requests against the ADR area of edgex-docs. Use the latest current ADR template to help create the ADR document.

"},{"location":"design/Process/#adr-review-and-approval-process","title":"ADR Review and Approval Process","text":"

Designs are created to address one or more requirements across one or more use cases. The design would include architecture details as well as references to pre-approved use cases and requirements. The TSC review the proposed design from a technical perspective. Approved designs get added to the EdgeX archive as \"approved\" records. They may get \"deprecated\" before implementation if another design supersedes it or if the requirements become obsolete over time. Designs may also get demoted if experimental implementations prove that they are not suitable (e.g. due to security, performance, dependency deprecation, feasibility). The design, implementation, verification cycles can repeat many times before resulting in a stable release.

"},{"location":"design/Process/#project-board-cards-and-issues","title":"Project Board Cards and Issues","text":"

All project design/architectural design decisions captured on the Design Decisions project board will be created as either a:

Issue: for any design decision that will require code and a PR will be submitted against the issue.\nCard: for any design decision that is not itself going to result in code or may need to be broken down into multiple issues (which can be referenced on the card).\n

The template for project board cards documenting each decision is:

When/Where: date of the decision and place where the decision was made (such as TSC meeting, working group meeting, etc.). This section is required.\nDecision Summary: quick write-up on the decision. This section is required.\nNotes/Considerations: any alternatives discussed, any impacts to other decisions or considerations to be considered in the future (which would negate the decision). This section is optional.\n\nRelevant links: link to the meeting recording (if available). Link to ADR if relevant. Link to PRs or Issues if relevant. Required if available.\n

Note there is a Template column on the project board with a single card that specifies this same structure.

"},{"location":"design/Process/#project-board-columns","title":"Project Board Columns","text":"

The Design Decisions project board will be permanent and never archived or deleted. For each release, a new column named for that release will be created to hold the decisions (in the form of cards or issues) for that release.

The release columns may be \"frozen\" at the end of a release, but should never be deleted so that all design decisions can be retained for the life of the project.

"},{"location":"design/Process/#ownership-and-cardissue-creation","title":"Ownership and Card/Issue Creation","text":"

The TSC chair, vice-chair and product manager will have overall responsibility for the Design Decision project board. These people will also be responsible for capturing any decisions from TSC meetings or the Monthly Architect\u2019s Meeting as cards/issues on the board.

Work Group chairs are responsible for adding new design decision cards/issues that come for their work group or related meetings.

"},{"location":"design/TOC/","title":"Use Cases and Design Records","text":""},{"location":"design/TOC/#use-case-records-ucrs","title":"Use Case Records (UCRs)","text":"

Note

UCRs are listed in alphabetical order by title.

Name/Link Short Description Bring Your Own Vault Use Case for bringing your own Vault Common Configuration Use Case for having Common configuration used by all EdgeX services Core Data Retention and Persistent Cap Use Case for capping readings in Core Data Device Parent-Child Relationships Use Case for Device Parent-Child Relationships Extending Device Data Use Case for Extending of Device Data by Application Services Provision Watch via Device Metadata Use Case for Provision Watching via Additional Device Metadata Record and Replay Use Case for Recording and Replaying event/readings System Events for Devices Use Case for System Events for Device add/update/delete Microservice Authentication Use Case for Microservice Authentication URIs for files Use Case for loading service files from URIs Performance Test Harness Use Case for Performance Test"},{"location":"design/TOC/#architectural-design-records-adrs","title":"Architectural Design Records (ADRs)","text":"

Note

ADRs are listed in chronological order by sequence number in title.

Name/Link Short Description 0001 Registry Refactor Separate out Registry and Configuration APIs 0002 Array Datatypes Allow Arrays to be held in Readings 0003 V2 API Principles Principles and Goals of V2 API Design 0004 Feature Flags Feature Flag Implementation 0005 Service Self Config Init Service Self Config Init & Config Seed Removal 0006 Metrics Collection Collection of service telemetry data 0007 Release Automation Overview of Release Automation Flow for EdgeX 0008 Secret Distribution Creation and Distribution of Secrets 0009 Secure Bootstrapping Secure Bootstrapping of EdgeX 0011 Device Service REST API The REST API for Device Services in EdgeX v2.x 0012 Device Service Filters Device Service event/reading filters 0013 Device Service Events via Message Bus Device Services send Events via Message Bus 0014 Secret Provider for All Secret Provider for All EdgeX Services 0015 Encryption between microservices Details conditions under which TLS is or is not used 0016 Container Image Guidelines Documents best practices for security of docker images 0017 Securing access to Consul Access control and authorization strategy for Consul 0018 Service Registry Service registry usage for EdgeX services 0019 EdgeX-CLI V2 EdgeX-CLI V2 Implementation 0020 Delay start services (SPIFFE/SPIRE) Secret store tokens for delayed start services 0021 Device Profile Changes Rules on device profile modifications 0022 Unit of Measure Unit of Measure 0023 North South Messaging Provide for messaging from north side systems through command down to device services 0024 System Events System Events (aka Control Plane Events) published to the MessageBus 0025 Record and Replay Record data from various devices and play data back without devices present 0026 Common Configuration Separate out the common configuration setting into a single source for all the services 0027 URIs for Files Add capability to load service files from remote locations using URIs 0028 Microservice communication security (token) Microservice communication security / authentication (token-based) 0029 Microservice communication security (E2EE) Microservice communication security / authentication (end-to-end authentication) 0030 Performance Test Harness Run Performance Test and generate report"},{"location":"design/adr/","title":"Architecture Decision Records Folder","text":"

This folder contains the EdgeX Foundry architectural decision records (ADR).

At the root of this folder are decisions that are relevant to multiple parts of the project (aka. cross cutting concerns). Sub folders under the ADR folder contain decisions relevant to the specific area of the project and essentially set up along working group lines (security, core, application, etc.).

"},{"location":"design/adr/#naming-and-formatting","title":"Naming and Formatting","text":"

ADR documents should follow the RFC (request for comments) naming standard. Specifically, approved ADRs should have a sequentially increasing integer (or serial number) and then the architectural design topic as file names (sequence_number-My-Topic.md). Example: 0001-Separate-Configuration-Interface. The sequence is a global sequence for all EdgeX ADR. Per RFC and Michael Nygard suggestions the makeup of the ADR document should generally include:

EdgeX ADRs should use the template.md file available in this directory.

"},{"location":"design/adr/#ownership","title":"Ownership","text":"

EdgeX WG chairman own the sub folder and included documents associated to their work group. The EdgeX TSC chair/vice chair are responsible for the root level, cross cutting concern documents.

"},{"location":"design/adr/#table-of-contents","title":"Table of Contents","text":"

A README with a table of contents for current documents is located here. Document authors are asked to keep the TOC updated with each new document entry.

Legacy designs have their own Table of Contents and are located here.

"},{"location":"design/adr/0001-Registy-Refactor/","title":"Registry Refactoring Design","text":""},{"location":"design/adr/0001-Registy-Refactor/#status","title":"Status","text":"

Approved

"},{"location":"design/adr/0001-Registy-Refactor/#context","title":"Context","text":"

Currently the Registry Client in go-mod-registry module provides Service Configuration and Service Registration functionality. The goal of this design is to refactor the go-mod-registry module for separation of concerns. The Service Registry functionality will stay in the go-mod-registry module and the Service Configuration functionality will be separated out into a new go-mod-configuration module. This allows for implementations for deferent providers for each, another aspect of separation of concerns.

"},{"location":"design/adr/0001-Registy-Refactor/#proposed-design","title":"Proposed Design","text":""},{"location":"design/adr/0001-Registy-Refactor/#provider-connection-information","title":"Provider Connection information","text":"

An aspect of using the current Registry Client is \"Where do the services get the Registry Provider connection information?\" Currently all services either pull this connection information from the local configuration file or from the edgex_registry environment variable. Device Services also have the option to specify this connection information on the command line. With the refactoring for separation of concerns, this issue changes to \"Where do the services get the Configuration Provider connection information?\"

There have been concerns voiced by some in the EdgeX community that storing this Configuration Provider connection information in the configuration which ultimately is provided by that provider is not the right design.

This design proposes that all services will use the command line option approach with the ability to override with an environment variable. The Configuration Provider information will not be stored in each service's local configuration file. The edgex_registry environment variable will be deprecated. The Registry Provider connection information will continue to be stored in each service's configuration either locally or from theConfiguration Provider same as all other EdgeX Client and Database connection information.

"},{"location":"design/adr/0001-Registy-Refactor/#command-line-option-changes","title":"Command line option changes","text":"

The new -cp/-configProvider command line option will be added to each service which will have a value specified using the format {type}.{protocol}://{host}:{port} e.g consul.http://localhost:8500. This new command line option will be overridden by the edgex_configuration_provider environment variable when it is set. This environment variable's value has the same format as the command line option value.

If no value is provided to the -cp/-configProvider option, i.e. just -cp, and no environment variable override is specified, the default value of consul.http://localhost:8500 will be used.

if -cp/-configProvider not used and no environment variable override is specified the local configuration file is used, as is it now.

All services will log the Configuration Provider connection information that is used.

The existing -r/-registry command line option will be retained as a Boolean flag to indicate to use the Registry.

"},{"location":"design/adr/0001-Registy-Refactor/#bootstrap-changes","title":"Bootstrap Changes","text":"

All services in the edgex-go mono repo use the new common bootstrap functionality. The plan is to move this code to a go module for the Device Service and App Functions SDKs to also use. The current bootstrap modules pkg/bootstrap/configuration/registry.go and pkg/bootstrap/container/registry.go will be refactored to use the new Configuration Client and be renamed appropriately. New bootstrap modules will be created for using the revised version of Registry Client . The current use of useRegistry and registryClient for service configuration will be change to appropriate names for using the new Configuration Client. The current use of useRegistry and registryClient for service registration will be retained for service registration. Call to the new Unregister() API will be added to shutdown code for all services.

"},{"location":"design/adr/0001-Registy-Refactor/#config-seed-changes","title":"Config-Seed Changes","text":"

The conf-seed service will have similar changes for specifying the Configuration Provider connection information since it doesn't use the common bootstrap package. Beyond that it will have minor changes for switching to using the Configuration Client interface, which will just be imports and appropriate name refactoring.

"},{"location":"design/adr/0001-Registy-Refactor/#config-endpoint-changes","title":"Config Endpoint Changes","text":"

Since the Configuration Provider connection information will no longer be in the service's configuration struct, the config endpoint processing will be modified to add the Configuration Provider connection information to the resulting JSON create from service's configuration.

"},{"location":"design/adr/0001-Registy-Refactor/#client-interfaces-changes","title":"Client Interfaces changes","text":""},{"location":"design/adr/0001-Registy-Refactor/#current-registry-client","title":"Current Registry Client","text":"

This following is the current Registry Client Interface

type Client interface {\nRegister() error\nHasConfiguration() (bool, error)\nPutConfigurationToml(configuration *toml.Tree, overwrite bool) error\nPutConfiguration(configStruct interface{}, overwrite bool) error\nGetConfiguration(configStruct interface{}) (interface{}, error)\nWatchForChanges(updateChannel chan<- interface{}, errorChannel chan<- error, configuration interface{}, waitKey string)\nIsAlive() bool\nConfigurationValueExists(name string) (bool, error)\nGetConfigurationValue(name string) ([]byte, error)\nPutConfigurationValue(name string, value []byte) error\nGetServiceEndpoint(serviceId string) (types.ServiceEndpoint, error)\nIsServiceAvailable(serviceId string) error\n}\n
"},{"location":"design/adr/0001-Registy-Refactor/#new-configuration-client","title":"New Configuration Client","text":"

This following is the new Configuration Client Interface which contains the Service Configuration specific portion from the above current Registry Client.

type Client interface {\nHasConfiguration() (bool, error)\nPutConfigurationFromToml(configuration *toml.Tree, overwrite bool) error\nPutConfiguration(configStruct interface{}, overwrite bool) error\nGetConfiguration(configStruct interface{}) (interface{}, error)\nWatchForChanges(updateChannel chan<- interface{}, errorChannel chan<- error,\nconfiguration interface{}, waitKey string)\nIsAlive() bool\nConfigurationValueExists(name string) (bool, error)\nGetConfigurationValue(name string) ([]byte, error)\nPutConfigurationValue(name string, value []byte) error\n}\n
"},{"location":"design/adr/0001-Registy-Refactor/#revised-registry-client","title":"Revised Registry Client","text":"

This following is the revised Registry Client Interface, which contains the Service Registry specific portion from the above current Registry Client. The UnRegister() API has been added per issue #20

type Client interface {\nRegister() error\nUnRegister() error\nIsAlive() bool\nGetServiceEndpoint(serviceId string) (types.ServiceEndpoint, error)\nIsServiceAvailable(serviceId string) error\n}\n
"},{"location":"design/adr/0001-Registy-Refactor/#client-configuration-structs","title":"Client Configuration Structs","text":""},{"location":"design/adr/0001-Registy-Refactor/#current-registry-client-config","title":"Current Registry Client Config","text":"

The following is the current struct used to configure the current Registry Client

type Config struct {\nProtocol string\nHost string\nPort int\nType string\nStem string\nServiceKey string\nServiceHost string\nServicePort int\nServiceProtocol string\nCheckRoute string\nCheckInterval string\n}\n
"},{"location":"design/adr/0001-Registy-Refactor/#new-configuration-client-config","title":"New Configuration Client Config","text":"

The following is the new struct the will be used to configure the new Configuration Client from the command line option or environment variable values. The Service Registry portion has been removed from the above existing Registry Client Config

type Config struct {\nProtocol string\nHost string\nPort int\nType string\nBasePath string\nServiceKey string\n}\n
"},{"location":"design/adr/0001-Registy-Refactor/#new-registry-client-config","title":"New Registry Client Config","text":"

The following is the revised struct the will be used to configure the new Registry Client from the information in the service's configuration. This is mostly unchanged from the existing Registry Client Config, except that the Stem for configuration has been removed

type Config struct {\nProtocol string\nHost string\nPort int\nType string\nServiceKey string\nServiceHost string\nServicePort int\nServiceProtocol string\nCheckRoute string\nCheckInterval string\n}\n
"},{"location":"design/adr/0001-Registy-Refactor/#provider-implementations","title":"Provider Implementations","text":"

The current Consul implementation of the Registry Client will be split up into implementations for the new Configuration Client in the new go-mod-configuration module and the revised Registry Client in the existing go-mod-registry module.

"},{"location":"design/adr/0001-Registy-Refactor/#decision","title":"Decision","text":"

It was decided to move forward with the above design

After initial ADR was approved, it was decided to retain the -r/--registry command-line flag and not add the Enabled field in the Registry provider configuration.

"},{"location":"design/adr/0001-Registy-Refactor/#consequences","title":"Consequences","text":"

Once the refactoring of go-mod-registry and go-mod-configuration are complete, they will need to be integrated into the new go-mod-bootstrap. Part of this integration will be the Command line option changes above. At this point the edgex-go services will be integrated with the new Registry and Configuration providers. The App Services SDK and Device Services SDK will then need to integrate go-mod-bootstrap to take advantage of these new providers.

"},{"location":"design/adr/0001-Registy-Refactor/#references","title":"References","text":"

Registry Abstraction - Decouple EdgeX services from Consul (Previous design)

"},{"location":"design/adr/0004-Feature-Flags/","title":"Feature Flag Proposal","text":""},{"location":"design/adr/0004-Feature-Flags/#status","title":"Status","text":"

Accepted

"},{"location":"design/adr/0004-Feature-Flags/#context","title":"Context","text":"

Out of the proposal for releasing on time, the community suggested that we take a closer look at feature-flags.

Feature-flags are typically intended for users of an application to turn on or off new or unused features. This gives user more control to adopt a feature-set at their own pace \u2013 i.e disabling store and forward in App Functions SDK without breaking backward compatibility.

It can also be used to indicate to developers the features that are more often used than others and can provided valuable feedback to enhance and continue a given feature. To gain that insight of the use of any given feature, we would require not only instrumentation of the code but a central location in the cloud (i.e a TIG stack) for the telemetry to be ingested and in turn reported in order to provide the feedback to the developers. This becomes infeasible primarily because the cloud infrastructure costs, privacy concerns, and other unforeseen legal reasons for sending \u201cUsage Metrics\u201d of an EdgeX installation back to a central entity such as the Linux Foundation, among many others. Without the valuable feedback loop, feature-flags don\u2019t provide much value on their own and they certainly don\u2019t assist in increasing velocity to help us deliver on time.

Putting aside one of the major value propositions listed above, feasibility of a feature flag \u201cmodule\u201d was still evaluated. The simplest approach would be to leverage configuration following a certain format such as FF_[NewFeatureName]=true/false. This is similar to what is done today. Turning on/off security is an example, turning on/off the registry is another. Expanding this further with a module could offer standardization of controlling a given feature such as featurepkg.Register(\u201cMyNewFeature\u201d) or featurepkg.IsOn(\u201cMyNewFeature\u201d). However, this really is just adding complexity on top of the underlying configuration that is already implemented. If we were to consider doing something like this, it lends it self to a central management of features within the EdgeX framework\u2014either its own service or possibly added as part of the SMA. This could help address concerns around feature dependencies and compatibility. Feature A on Service X requires Feature B and Feature C on Service Y. Continuing down this path starts to beget a fairly large impact to EdgeX for value that cannot be fully realized.

"},{"location":"design/adr/0004-Feature-Flags/#decision","title":"Decision","text":"

The community should NOT pursue a full-fledged feature flag implementation either homegrown or off-the-shelf.

However, it should be encouraged to develop features with a wholistic perspective and consider leveraging configuration options to turn them on/off. In other words, once a feature compiles, can work under common scenarios, but perhaps isn\u2019t fully tested with edge cases, but doesn\u2019t impact any other functionality, should be encouraged.

"},{"location":"design/adr/0004-Feature-Flags/#consequences","title":"Consequences","text":"

Allows more focus on the many more competing priorities for this release.

Minimal impact to development cycles and release schedule

"},{"location":"design/adr/0005-Service-Self-Config/","title":"Service Self Config Init & Config Seed Removal","text":""},{"location":"design/adr/0005-Service-Self-Config/#status","title":"Status","text":"

approved - TSC vote on 3/25/20 for Geneva release

NOTE: this ADR does not address high availability considerations and concerns. EdgeX, in general, has a number of unanswered questions with regard to HA architecture and this design adds to those considerations.

"},{"location":"design/adr/0005-Service-Self-Config/#context","title":"Context","text":"

Since its debut, EdgeX has had a configuration seed service (config-seed) that, on start of EdgeX, deposits configuration for all the services into Consul (our configuration/registry service). For development purposes, or on resource constrained platforms, EdgeX can be run without Consul with services simply reading configuration from the filesystem.

While this process has nominally worked for several releases of EdgeX, there has always been some issues with this extra initialization process (config-seed), not least of which are: - race conditions on the part of the services, as they bootstrap, coming up before the config-seed completes its deposit of configuration into Consul - how to deal with \"overrides\" such as environmental variable provided configuration overrides. As the override is often specific to a service but has to be in place for config-seed in order to take effect. - need for an additional service that is only there for init and then dies (confusing to users)

NOTE - for historical purposes, it should be noted that config-seed only writes configuration into the configuration/registry service (Consul) once on the first start of EdgeX. On subsequent starts of EdgeX, config-seed checks to see if it has already populated the configuration/registry service and will not rewrite configuration again (unless the --overwrite flag is used).

The design/architectural proposal, therefore, is: - removal of the config-seed service (removing cmd/config-seed from the edgex-go repository) - have each EdgeX micro service \"self seed\" - that is seed Consul with their own required configuration on bootstrap of the service. Details of that bootstrapping process are below.

"},{"location":"design/adr/0005-Service-Self-Config/#command-line-options","title":"Command Line Options","text":"

All EdgeX services support a common set of command-line options, some combination of which are required on startup for a service to interact with the rest of EdgeX. Command line options are not set by any configuration. Command line options include:

The distinction of command line options versus configuration will be important later in this ADR.

Two command line options (-o for overwrite and -r for registry) are not overridable by environmental variables.

NOTES: Use of the --overwrite command line option should be used sparingly and with expert knowledge of EdgeX; in particular knowledge of how it operates and where/how it gets its configuration on restarts, etc. Ordinarily, --overwrite is provided as a means to support development needs. Use of --overwrite permanently in production enviroments is highly discouraged.

"},{"location":"design/adr/0005-Service-Self-Config/#configuration-initialization","title":"Configuration Initialization","text":"

Each service has (or shall have if not providing it already) a local configuration file. The service may use the local configuration file on initialization of the service (aka bootstrap of the service) depending on command line options and environmental variables (see below) provided at startup.

Using a configuration provider

When the configuration provider is specified, the service will call on the configuration provider (Consul) and check if the top-level (root) namespace for the service exists. If configuratation at the top-level (root) namespace exists, it indicates that the service has already populated its configuration into the configuration provider in a prior startup.

If the service finds the top-level (root) namespace is already populated with configuration information it will then read that configuration information from the configuration provider under namespace for that service (and ignore what is in the local configuration file).

If the service finds the top-level (root) namespace is not populated with configuration information, it will read its local configuration file and populate the configuration provider (under the namespace for the service) with configuration read from the local configuration file.

A configuration provider can be specified with a command line argument (the -cp / --configProvider) or environment variable (the EDGEX_CONFIGURATION_PROVIDER environmental variable which overrides the command line argument).

NOTE: the environmental variables are typically uppercase but there have been inconsistencies in environmental variable casing (example: edgex_registry). This should be considered and made consistent in a future major release.

Using the local configuration file

When a configuration provider isn't specified, the service just uses the configuration in its local configuration file. That is the service uses the configuration in the file associated with the profile, config filename and config file directory command line options or environmental variables. In this case, the service does not contact the configuration service (Consul) for any configuration information.

NOTE: As the services now self seed and deployment specific changes can be made via environment overrides, it will no longer be necessary to have a Docker profile configuration file in each of the service directories (example: https://github.com/edgexfoundry/edgex-go/blob/master/cmd/core-data/res/docker/configuration.toml). See Consequences below. It will still be possible for users to use the profile mechanism to specify a Docker configuration, but it will no longer be required and not the recommended approach to providing Docker container specific configuration.

"},{"location":"design/adr/0005-Service-Self-Config/#overrides","title":"Overrides","text":"

Environment variables used to override configuration always take precedence whether configuration is being sourced locally or read from the config provider/Consul.

Note - this means that a configuration value that is being overridden by an environment variable will always be the source of truth, even if the same configuration is changed directly in Consul.

The name of the environmental variable must match the path names in Consul.

NOTES: - Environmental variables overrides remove the need to change the \"docker\" profile in the res/docker/configuration.toml files - Allowing removal of 50% of the existing configuration.toml files. - The override rules in EdgeX between environmental variables and command line options may be counter intuitive compared to other systems. There appears to be no standard practice. Indeed, web searching \"Reddit & Starting Fights Env Variables vs Command Line Args\" will layout the prevailing differences. - Environment variables used for configuration overrides are named by prepending the the configuration element with the configuration section inclusive of sub-path, where sub-path's \".\"s are replaced with underscores. These configuration environment variable overrides must be specified using camel case. Here are two examples: 

Registry_Host  for\n[Registry]\nHost = 'localhost'\n\nClients_CoreData_Host for\n[Clients]\n  [Clients.CoreData]\n  Host = 'localhost'\n
- Going forward, environmental variables that override command line options should be all uppercase.

All values overriden get logged (indicating which configuration value or op param and the new value).

"},{"location":"design/adr/0005-Service-Self-Config/#decision","title":"Decision","text":"

These features have been implemented (with some minor changes to be done) for consideration here: https://github.com/edgexfoundry/go-mod-bootstrap/compare/master...lenny-intel:SelfSeed2. This code branch will be removed once this ADR is approved and implemented on master.

The implementation for self-seeding services and environmental overrides is already implemented (for Fuji) per this document in the application services and device services (and instituted in the SDKs of each).

"},{"location":"design/adr/0005-Service-Self-Config/#backward-compatibility","title":"Backward compatibility","text":"

Several aspects of this ADR contain backward compatibility issues for the device service and application service SDKs. Therefore, for the upcoming minor release, the following guidelines and expections are added to provide for backward compatibility.