Sense Collector is a Python application deployed with Docker designed to efficiently collect data from the Sense energy monitoring system. Once deployed, Sense Collector provides a comprehensive set of Grafana dashboards, enabling you to visualize and effortlessly analyze your energy usage data in real-time. Whether starting with Grafana, InfluxDB, and Sense or looking to enhance your existing setup, Sense Collector offers an intuitive and powerful solution for monitoring and understanding your energy consumption.
A live set of these dashboards are available to view.
The project is deployed as a Docker container.
- Docker
- Docker Compose
- InfluxDB 2.x (The latest version of the Sense Collector no longer works with InfluxDB V1)
- Grafana 11.0.0
Use the following Docker container:
lux4rd0/sense-collector:2.0.07
lux4rd0/sense-collector:latest
Correct environmental variables are required for the container to function.
SENSE_COLLECTOR_API_PASSWORD
SENSE_COLLECTOR_API_USERNAME
SENSE_COLLECTOR_INFLUXDB_BUCKET
SENSE_COLLECTOR_INFLUXDB_ORG
SENSE_COLLECTOR_INFLUXDB_TOKEN
SENSE_COLLECTOR_INFLUXDB_URL
The username and password are the same as those used in the Sense mobile or Sense Web apps.
An example command line would be (be sure to change all of the variables):
To start the docker container, simply update this minimal example compose.yaml file:
name: sense-collector-53997
services:
sense-collector-53997:
container_name: sense-collector-53997
environment:
SENSE_COLLECTOR_API_PASSWORD: CHANGEME
SENSE_COLLECTOR_API_USERNAME: dave@lux4rd0.com
SENSE_COLLECTOR_INFLUXDB_BUCKET: sense
SENSE_COLLECTOR_INFLUXDB_ORG: Lux4rd0
SENSE_COLLECTOR_INFLUXDB_TOKEN: TOKEN
SENSE_COLLECTOR_INFLUXDB_URL: http://sense-collector.lux4rd0.com:8086
TZ: America/Chicago
image: lux4rd0/sense-collector:latest
restart: always
If you don't want to use docker-compose, an example docker run command will be displayed on the screen.
docker run --rm \
--name=sense-collector-53997 \
-e SENSE_COLLECTOR_API_PASSWORD=CHANGEME \
-e SENSE_COLLECTOR_API_USERNAME=dave@lux4rd0.com \
-e SENSE_COLLECTOR_INFLUXDB_BUCKET=sense \
-e SENSE_COLLECTOR_INFLUXDB_ORG=Tylephony \
-e SENSE_COLLECTOR_INFLUXDB_TOKEN=TOKEN \
-e SENSE_COLLECTOR_INFLUXDB_URL=http://sense-collector.lux4rd0.com:8086 \
-e TZ=America/Chicago \
--restart always \
lux4rd0/sense-collector:latest
Running docker compose up -d or the docker-run command will download and start up the sense-collector container.
The Docker contain can be configured with additional environment flags to control collector behaviors. They are described below:
This document provides a detailed explanation of the environmental variables used to configure the Sense Collector Docker container. Each variable is classified as required or optional, with its default value and possible options.
- Description: The password for authenticating with the Sense API.
- Required: Yes
- Default: None
- Options: User-provided password
- Description: The username (email address) for authenticating with the Sense API.
- Required: Yes
- Default: None
- Options: User-provided email address
- Description: Interval (in seconds) renews the Sense API token to maintain an active session.
- Required: No
- Default: 43200 (12 hours)
- Options: Integer value representing seconds
- Description: The hostname that is running the Docker container. Used in the Collector Info dashboard to know where the Collector is running.
- Required: No
- Default: None
- Options: User-provided hostname
- Description: The bucket name in InfluxDB where data will be stored.
- Required: Yes
- Default: None
- Options: User-provided bucket name
- Description: The organization name in InfluxDB.
- Required: Yes
- Default: None
- Options: User-provided organization name
- Description: The authentication token for InfluxDB.
- Required: Yes
- Default: None
- Options: User-provided token
- Description: The URL of the InfluxDB instance where data will be written.
- Required: Yes
- Default: None
- Options: User-provided URL
- Description: Sets the logging level for the API interactions.
- Required: No
- Default: INFO
- Options: DEBUG, INFO, WARNING, ERROR, CRITICAL
- Description: Sets the general logging level for the application.
- Required: No
- Default: INFO
- Options: DEBUG, INFO, WARNING, ERROR, CRITICAL
- Description: Sets the logging level for storage operations.
- Required: No
- Default: INFO
- Options: DEBUG, INFO, WARNING, ERROR, CRITICAL
- Description: Enables or disables the output of received Sense API data to a file.
- Required: No
- Default: false
- Options: true, false
- Description: Interval (in seconds) between heartbeat messages to keep the WebSocket connection alive.
- Required: No
- Default: 10
- Options: Integer value representing seconds
- Description: Timeout (in seconds) to wait for a heartbeat response before considering the WebSocket connection as dead.
- Required: No
- Default: 30
- Options: Integer value representing seconds
- Description: Initial delay (in seconds) before reconnecting the WebSocket after a connection loss.
- Required: No
- Default: 5
- Options: Integer value representing seconds
- Description: Maximum delay (in seconds) between reconnection attempts.
- Required: No
- Default: 60
- Options: Integer value representing seconds
- Description: Maximum number of retries for reconnecting the WebSocket before giving up.
- Required: No
- Default: 3
- Options: Integer value
- Description: Factor for exponential backoff between reconnection attempts.
- Required: No
- Default: 1
- Options: Integer value
- Description: Interval (in seconds) between forced WebSocket reconnections to maintain a fresh connection.
- Required: No
- Default: 840
- Options: Integer value representing seconds
- Description: Duration (in seconds) for which the device data should be cached before it expires.
- Required: No
- Default: 120
- Options: Integer value representing seconds
- Description: Maximum number of concurrent lookups that can be performed for device data.
- Required: No
- Default: 4
- Options: Integer value
- Description: Delay (in seconds) between consecutive device data lookup requests.
- Required: No
- Default: 0.5
- Options: Float value representing seconds
Sense Collector is the primary data collector and is responsible for gathering details on the following:
Voltage, Watts, Hz on the mains, and device-specific wattage details are collected. If you happen to have any Sense compatible smart plugs, their additional metric details of voltage and amps are also collected.
Timeline events for device's change of states (on, off, idle)
Device Details polls the Sense API to gather details on your devices. This includes:
avg_duration, avg_monthly_KWH, avg_monthly_cost, avg_monthly_cost, avg_monthly_pct, avg_monthly_runs, avg_watts, current_ao_wattage, current_month_KWH, current_month_cost, current_month_runs, icon, last_state, last_state_time, name, yearly_KWH, yearly_cost, yearly_text
Monitor Status gathers details about the Sense monitor and detection status for both in-progress and found devices. Monitor details include: emac, ethernet, ip_address, mac, ndt_enabled, online, progress, serial, signal, ssid, status, test_result, version, wifi_strength
Collecting data is only half the fun. Now it's time to provision some Grafana Dashboards to visualize your essential Sense data. You'll find a folder of dashboards with collectors and backends split out. You can also use the links/numbers next to each dashboard title to load the dashboards directly from Grafana.
Each dashboard has dropdowns at the top that allow you to filter measurements based on devices and plugs. The dropdowns default to "All," but you can select and save preferences.
Interval: A dropdown that provides different smoothing levels helps manage how the graphs look based on the interval of data collected by the Sense Collector. Think of this as a level of "smoothing" based on your chosen time frame and the quantity of data collected.
Device Status On/Off/Idle: These three toggles overlay event annotations in the Wattage By Devices panels and the Wattage, Volts, and Hz panels in the Mains Overview dashboard.
Sense Collector Dashboard Links: This dropdown in the top right-hand corner provides easy access to the other dashboards in this collection.
Time Range: This defaults to "Today so far" but can be updated to any other Relative or Absolute time range. Change the "Interval" dropdown for longer time ranges to smooth out any of the data.
Dashboard Refresh: Each dashboard is set to refresh every sixty seconds, but this can be changed or disabled.
Collector Info - 14734
Collector Info: This dashboard provides observability into how the Sense Collector functions alongside metrics related to the host's performance. It helps understand the performance of the main collector functions to assist with troubleshooting.
Epoch Time Difference: Helps determine if your hosts can keep up with processing messages from the Sense monitor. It provides the difference between the host and epoch times received in the Sense monitor data. Negative numbers mean the Sense monitor is ahead of the hosts. Positive numbers mean the host is behind the Sense monitor. If the drift trends positively, it may also mean there's just time clock drift. Ensure you monitor the NTP time sync on both your host and Sense if there's a large discrepancy.
CPU, Load Average, Memory Utilization: These panels show host-level details and are not specific to the performance of the docker container. Per Process CPU Usage, Netstat, and Processes are particular to the container.
Collector Starts: Provide the last time the container and process were started. Because Sense times out the connection, it is reset every fifteen minutes.
Device Overview - 14735
Device Overview is the main dashboard for Sense Collector. Here, you'll see several sections about the device's overall details.
Current Wattage: A Bubble Chart showing current wattage usage by device. Larger circles represent higher wattage consumption.
Wattage By Device (Stacked): Wattage overtime per device. The graph is stacked to represent total household wattage.
Device Status: This is a State Timeline representing event data from the Sensor Monitor over time. It currently represents three states: On, Off, and Idle.
Event Timeline: This table view shows each device's state change.
Device Details - Average: This is a table view of current data representing the current state, state time, watts in use, average, and always on makeup. Average duration, monthly kWh, percent, and the number of runs are also listed. This table defaults to being sorted by "Watts (In Use)."
Device Details - Current Month: Shows the number of runs per device since the start of the current month.
Device Details - Yearly: This section shows each device's calculated kWh and costs. The text next to each device provides details on how the costs are calculated, such as "Based on the last 30 days," "Based on the last 7 days," or "Based on last season."
Smart Device Details: If you have any of the Sense compatible smart plugs, they will be listed here as well. Open up this row to display the voltages and amps measured by each plug. There's another measurement (Who Knows By Plugs), but I don't know what it does yet.
Always On Devices: This shows which devices the Sense monitor has detected have an Always On wattage component. This may be different from actual wattage and tends to update less frequently.
Notice: This Grafana dashboard uses the community visualization Bubble Chart panel plugin.
Mains Overview - 14736
Mains Overview provides three panels showing Wattage (Stacked), Voltages, and Frequency. Dropdowns at the top of the dashboard show Leg 1, Leg 2, or both together. Device Status On, Off, and Idle event annotations may be toggled on or off.
Monitor & Detection - 14737
The Monitor & Detection dashboard provides observability of the monitor itself.
Device Detection Status: Represents Device Detection Status for both "Found" and "In Progress" devices.
Wifi Signal Strength - RSSI: Represents the Wifi signal strength of your Sense monitor.
Monitor Details: This panel shows current information about Online status, General Status, Learning Progress, IP Address, MAC Address, Wi-Fi SSID, Wi-Fi strength, Ethernet, NDT Enabled, and software version.
This collector uses InfluxQL, and for the dashboards to function, you need to create a data source in Grafana using the credentials you set in InfluxDB V2. More details can be found on the InfluxDB V2 Web site:
https://docs.influxdata.com/influxdb/v2/tools/grafana/?t=InfluxQL#configure-your-influxdb-connection
The biggest change here is:
-
Configure InfluxDB authentication:
Token authentication Under Custom HTTP Headers, select Add Header. Provide your InfluxDB API token:
Header: Enter
AuthorizationValue: Use the
Tokenschema and provide your InfluxDB API token. For example:Token y0uR5uP3rSecr3tT0k3n
The Sense Collector and dashboards currently only support a single Sense instance. Please let me know if you have more than one Sense device and are interested in having multiple devices supported.
A TZ variable is used when running the Docker container based on the value returned during the initial docker-compose configuration scripts. It's not currently in use in any of the current dashboards.
See the open issues for a list of proposed features (and known issues).
Dave Schmid: @lux4rd0 - dave@pulpfree.org
Project Link: https://github.com/lux4rd0/sense-collector
- Grafana Labs - https://grafana.com/
- Grafana - https://grafana.com/oss/grafana/
- Grafana Dashboard Community - https://grafana.com/grafana/dashboards/