Skip to content
This repository has been archived by the owner on Oct 4, 2021. It is now read-only.

Add SM100 EMS-Plus device #96

Closed
S-Przybylski opened this issue Apr 15, 2019 · 77 comments
Closed

Add SM100 EMS-Plus device #96

S-Przybylski opened this issue Apr 15, 2019 · 77 comments
Labels
enhancement New feature or request

Comments

@S-Przybylski
Copy link

Is your feature request related to a problem? Please describe.
I want to investigate the EMS and EMS-Plus messages for my solar module SM100 device in future.

Describe the solution you'd like
Can you please update the device table for my SM100?
...
0x30 -> 0x09, type 0x02 telegram: 30 09 02 00 A3 15 04 (CRC=25), #data=3
<--- Version(0x02) received
Unrecognized device found. TypeID 0x02, ProductID 163, Version 21.04

Additional context
I assume that the implementation of SM10 is not true generic one. My current log shows messages of a SM10, but in opposite direction the device is unknown... (i do not have a SM10 connected):
(00:25:56.914) 0x09 -> SM10, type 0x96 telegram: 09 B0 96 00 07 (CRC=00), #data=1
(00:25:56.967) 0x30 -> 0x09, type 0x96 telegram: 30 09 96 00 FF 18 1E 0A 02 50 28 (CRC=B5), #data=7
(00:25:56.979) 0x09 -> SM10, type 0x00 telegram: 09 B0 FF 00 FF 00 01 (CRC=0C), #data=3

The autodetect shows the following:
Started scan on EMS bus for known devices
Boiler found. Model Buderus GB172/Nefit Trendline (TypeID:0x08 ProductID:123 Version:04.09)
Device found. Model BC25 Base Controller with TypeID 0x09, ProductID 125, Version 01.06
Unrecognized device found. TypeID 0x02, ProductID 163, Version 21.04

@S-Przybylski S-Przybylski added the enhancement New feature or request label Apr 15, 2019
@proddy
Copy link
Collaborator

proddy commented Apr 15, 2019

could you try the latest dev build (under dev branch) and do a autodetect deep to see if your SM is correctly detected and configured?

@S-Przybylski
Copy link
Author

  • Connected to: EMS-ESP version 1.7.0b5
  • autodetect deep
    Started scan on EMS bus for known devices
    Boiler found. Model Buderus GB172/Nefit Trendline (TypeID:0x08 ProductID:123 Version:04.09)
    Device found. Model BC25 Base Controller with TypeID 0x09, ProductID 125, Version 01.06
    Unrecognized device found. TypeID 0x02, ProductID 163, Version 21.04
    Unrecognized device found. TypeID 0x02, ProductID 163, Version 21.04
    Read operations not yet supported for this model thermostat
    Read operations not yet supported for this model thermostat

@proddy
Copy link
Collaborator

proddy commented Apr 16, 2019 via email

@S-Przybylski
Copy link
Author

  • Connected to: EMS-ESP version 1.7.0b6
  • autodetect deep
    System Logging set to None
    Starting a deep EMS device scan. This will take about 1 minute. Please wait...
    Device found. Model BC25 Base Controller with TypeID 0x09, ProductID 125, Version 01.06
    Device found. Model BC25 Base Controller with TypeID 0x09, ProductID 125, Version 01.06
    Finished the deep EMS device scan.

These 4 EMS devices were detected:
Buderus GB172/Nefit Trendline (TypeID:0x08 ProductID:123 Version:04.09)
SM100 Solar Module (TypeID:0x30 ProductID:163 Version:21.04)
RC310 (TypeID:0x64 ProductID:158 Version:11.07)
BC25 Base Controller (TypeID:0x09 ProductID:125 Version:01.06)

The log shows the following:

  • It seems that if the solar module is addressed the value SM is used, but the 0x30 address will not be identified as SM100
    (00:09:14.444) 0x30 -> Boiler, type 0x33 telegram: 30 88 33 02 07 (CRC=C0), #data=1
    <--- UBAParameterWW(0x33) received
    (00:09:14.472) Boiler -> SM, type 0x33 telegram: 08 30 33 02 33 FB 00 1E FF 06 46 (CRC=C9), #data=7
    <--- UBAParameterWW(0x33) received
    (00:09:14.500) 0x30 -> Boiler, type 0x35 telegram: 30 08 35 02 00 (CRC=BB), #data=1
    (00:09:14.625) 0x30 -> all, type 0x00 telegram: 30 00 FF 00 02 62 01 9F 01 5D 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=2A), #data=26
    (00:09:15.061) 0x30 -> all, type 0x18 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE), #data=4
    (00:09:15.277) 0x30 -> all, type 0x00 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74), #data=15
    (00:09:15.724) 0x30 -> all, type 0x00 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 00 0C 64 00 00 00 00 (CRC=B8), #data=19
    (00:09:15.929) 0x30 -> all, type 0x00 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73), #data=6

info shows:

  • Solar Module stats:
    Collector temperature: 0.0 C
    Bottom temperature: 0.0 C
    Pump modulation: 0 %
    Pump active: off

In my installation a RC300 is installed. It seems to use the same identication like the RC310

@proddy
Copy link
Collaborator

proddy commented Apr 16, 2019 via email

@proddy
Copy link
Collaborator

proddy commented Apr 18, 2019

@S-Przybylski we can start to implement this but I need you to decode the messages. Look at the telegrams your getting

30 00 FF 00 02 62 01 9F 01 5D 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00
30 00 FF 18 02 62 80 00
30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00
30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 00 0C 64 00 00 00 00

The types sent from the Solar module are x0262, x0263 and x0264

See if you can correlate the temperature settings you see on your boiler/thermostat/mobile app to any of these values.

@proddy
Copy link
Collaborator

proddy commented Apr 19, 2019

I added support for type x0262 but need your help to decypher the data. We'll be looking for two temperatures and single byte modulation %

@S-Przybylski
Copy link
Author

Hi @proddy, i will check this out

@S-Przybylski
Copy link
Author

My first try:
30 0 FF 0 2 62 1 FB 1 9E 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 2B
--> Solar Kollektor ist 50,7 gradC --> 1FB --> Databyte 1+2
--> Wasserspeicher unten ist 41,4 gradC --> 19E -->Databyte 3+4
30 0 FF 0 2 64 0 0 0 4 0 0 FF 0 0 1E 0C 20 64 0 0 0 0 E9
--> my collector area (Fläche) is 4 m2 --> possible Databyte 4 ?
--> current solar pump modulation ratio is 30% --> hex 1E --> Databyte 10 ?
My (other) current settings are - i can't identify by now:
Hysterese: on 10.0 K; 0ff 5.0 K
Max Speichertemp. 80grad C
Max Kollektortemp 120grad C
Min Kollektortemp 20grad C

@proddy
Copy link
Collaborator

proddy commented Apr 22, 2019

Nice work. It’ll be easy to add this in code. Shall I do this or do you want to try?

I expect the max/min temps will be in a separate telegram type as it seems this x0262 is the SM100 monitor.

Also which if these data values do you think are important to report out via mqtt?

@S-Przybylski
Copy link
Author

Please do it for me, then i will learn on this example how to integrate new functions (Can you please provide the changes to me?)
First of all i am interrested in Temperatures and the Solarpump. Perhaps the daily sum of provided engery in KWH. The rest is nice to know, but not critical for me...

@proddy
Copy link
Collaborator

proddy commented Apr 25, 2019

ok. So those two telegrams 0x262 and 0x264. Can you tell me if they are broadcast telegrams, i.e. sent out automatically every 10-60 seconds?

@S-Przybylski
Copy link
Author

Investigation today 22:25 CET time (no sun active):

  1. 0x262, 0x263, 0x264, 0x266, 0x268 and 0x26a is send every 60 seconds, but not excactly. The negative drift is 200ms - 500ms. In addition to that, the protocol 0x266 is send 30 seconds later twice.

  2. I found the energy earnings of today and in total (from beginning) in protocol 28E (which occured only one time in 15 minutes (not quite sure what the interval realy is):
    0x519 =1305 --> 1,3 kwh (display) - factor 1/1000 (is the todays value) and 0x75D3 = 30163 --> 3016,3 kwh (display) - factor 1/10 is the total value (over all) !

(06:03:15.579) 0x30 -> all, type 0x00 telegram: 30 0 FF 0 2 8E 0 0 0 0 0 0 5 19 0 0 75 D3 (CRC=77), #data=14

Could you please also put the energy earnings into the mqtt message?

  1. I rechecked the protocol 0x262 and 0x264:
    0x262 all values seems to match
    0x264 databyte four (EMS+) is currently null - i expect 4 square meter - so i was wrong:
    (05:55:19.676) 0x30 -> all, type 0x00 telegram: 30 0 FF 0 2 64 0 0 0 0 0 0 FF 0 0 0 0 0A 64 0 0 0 0 (CRC=54), #data=19

  2. I just wonder that the 'log v' mode print in case of an EMS+ message the offset as the type. That seems to be wrong:
    (05:55:18.531) 0x30 -> all, type 0x00 telegram: 30 0 FF 0 2 62 0 75 1 9C 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 (CRC=2B), #data=26
    For me it should be something like type 'EMS+ (FF) --> 0x262 '
    (05:55:18.922) 0x30 -> all, type 0x18 telegram: 30 0 FF 18 2 62 80 0 (CRC=AE), #data=4
    For me it should be something like type 'EMS+ (FF) --> 0x262 ' and not '0x18'! Here you can see that the offset is '0x18' due to the fact, that the maximum transmit size is fixed but the protocol 0x262 needs more characters.

@proddy
Copy link
Collaborator

proddy commented Apr 26, 2019

thanks. As for point 4, I fixed this a few days ago in the dev branch and also added the SM100 support as defined by you. Could you do a quick check?

@S-Przybylski
Copy link
Author

First of all thanks for your efforts! Great. Currently the SM-MQTT Messages toogles between a good and at least two bad states - see investigation below...

  1. I was surprised that today the 0x262 protocol seems to update also a single value at the beginning without repeating the all other values: (00:24:18.020) SM -> all, type 0x0262 telegram: 30 0 FF 0 2 62 1 CA (CRC=E6)
    It seems that your routine wait for a full message but only get this short one and send out: {"temp":45.8,"bottomtemp":-665.6,"pumpmodulation":0,"pump":"off"}

  2. Cosmetic: If the EMS+ message is null byte, the data cannot be 255 byte length: (00:23:35.979) SM -> 0x09, type 0x0001 telegram: 30 9 FF 0 0 1 (CRC=70), #data=255

  3. The SM10Monitor(0x97) seems to interfere with the SM100 monitor: (00:23:36.916) SM -> me, type 0x97 telegram: 30 0B 97 0 (CRC=82)
    <--- SM10Monitor(0x97) received
    Publishing SM10 data via MQTT {"temp":0,"bottomtemp":0,"pumpmodulation":0,"pump":"off"}

Note: I just create a long term logfile to make a more statisical and useful interpretation of the protocol... hopefully i will get some additional benefits out of it (if so i provide them later here)

More detailed SM-Protocol:
(00:23:35.863) SM -> 0x09, type 0x02 telegram: 30 9 2 0 A3 15 4 (CRC=25)
SM10 Solar Module support enabled.
(00:23:35.931) SM -> 0x09, type 0x96 telegram: 30 9 96 0 FF 18 1E 0A 2 50 28 (CRC=B5), #data=7
(00:23:35.979) SM -> 0x09, type 0x0001 telegram: 30 9 FF 0 0 1 (CRC=70), #data=255
(00:23:36.916) SM -> me, type 0x97 telegram: 30 0B 97 0 (CRC=82)
<--- SM10Monitor(0x97) received
Publishing SM10 data via MQTT {"temp":0,"bottomtemp":0,"pumpmodulation":0,"pump":"off"}
(00:23:48.629) SM -> 0x09, type 0x02 telegram: 30 9 2 0 A3 15 4 (CRC=25)
SM10 Solar Module support enabled.
(00:23:48.682) SM -> 0x09, type 0x96 telegram: 30 9 96 0 FF 18 1E 0A 2 50 28 (CRC=B5), #data=7
(00:23:48.737) SM -> 0x09, type 0x0001 telegram: 30 9 FF 0 0 1 (CRC=70), #data=255
(00:23:49.353) SM -> all, type 0x0266 telegram: 30 0 FF 0 2 66 1 62 0 12 (CRC=73)
(00:23:50.112) SM -> me, type 0x97 telegram: 30 0B 97 0 (CRC=82)
<--- SM10Monitor(0x97) received

(00:24:18.020) SM -> all, type 0x0262 telegram: 30 0 FF 0 2 62 1 CA (CRC=E6)
<--- SM100Monitor(0x262) received
Publishing SM10 data via MQTT {"temp":45.8,"bottomtemp":-665.6,"pumpmodulation":0,"pump":"off"}

(00:24:19.171) SM -> all, type 0x0262 telegram: 30 0 FF 0 2 62 1 CA 1 93 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 80 0 (CRC=BE), #data=23
<--- SM100Monitor(0x262) received
Publishing SM10 data via MQTT {"temp":45.8,"bottomtemp":40.3,"pumpmodulation":0,"pump":"off"}
(00:24:19.601) SM -> all, type 0x0262 telegram: 30 0 FF 18 2 62 80 0 (CRC=AE)
<--- SM100Monitor(0x262) received
Publishing SM10 data via MQTT {"temp":"?","bottomtemp":-2099.2,"pumpmodulation":0,"pump":"off"}

(00:24:19.832) SM -> all, type 0x0263 telegram: 30 0 FF 0 2 63 80 0 80 0 0 0 80 0 80 0 80 0 0 (CRC=74), #data=12
Publishing SM10 data via MQTT {"temp":"?","bottomtemp":-2099.2,"pumpmodulation":0,"pump":"off"}

(00:24:20.280) SM -> all, type 0x0264 telegram: 30 0 FF 0 2 64 0 0 0 4 0 0 FF 0 0 1E 0B 9 64 0 0 0 0 (CRC=6D), #data=16
<--- SM100Status(0x264) received
Publishing SM10 data via MQTT {"temp":"?","bottomtemp":-2099.2,"pumpmodulation":30,"pump":"off"}

(00:24:20.482) SM -> all, type 0x0266 telegram: 30 0 FF 0 2 66 1 62 0 12 (CRC=73)
(00:24:21.237) SM -> all, type 0x0268 telegram: 30 0 FF 0 2 68 0C 0 (CRC=1E)
(00:24:21.451) SM -> all, type 0x026A telegram: 30 0 FF 0 2 6A 3 3 3 0 3 3 3 3 3 0 4 3 (CRC=EB), #data=11

@S-Przybylski
Copy link
Author

Details about states of the pump and changes:
Conclusion:
Type 0x026a databyte 0A: "03" solar pump off and "04" solar pump on
Short messages are send when the status of the pump changes
And as already said: The temperature of the collector is send in addition as short message only if the pump is on

Details:

  1. Status pump is off: see penultimate data byte "03" of 0x026a and data byte 9 of 0x264 = "00"
    SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5), #data=11
    SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 0C 0A 64 00 00 00 00 (CRC=53), #data=16

  2. Status pump is on; see penultimate data byte "04" of 0x026a and data byte 9 of 0x264 = "1E"
    SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 04 03 (CRC=EB), #data=11
    SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 1E 09 08 64 00 00 00 00 (CRC=CD), #data=16
    Note: If on, you get additional status updates of the collector temp as a short message: SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 02 1A (CRC=30)

  3. Solar pump changes from off --> on: type 0x0264 data byte 09 set first to "64" and type 0x026a byte 0A is set to "04"
    SM -> all, type 0x0264 telegram: 30 00 FF 09 02 64 64 (CRC=37)
    SM -> all, type 0x026A telegram: 30 00 FF 0A 02 6A 04 (CRC=53)
    less then 10 seconds later the pump is throttled to 30% 0x1E:
    SM -> all, type 0x0264 telegram: 30 00 FF 09 02 64 1E (CRC=4D)

  4. Solar pump changes from on --> off: type 0x0264 data byte 09 set to "00" and type 0x026a byte 0A is set to "03"
    SM -> all, type 0x0264 telegram: 30 00 FF 09 02 64 00 (CRC=53)
    SM -> all, type 0x026A telegram: 30 00 FF 0A 02 6A 03 (CRC=54)

@proddy
Copy link
Collaborator

proddy commented Apr 29, 2019

Questions so I can implement these changes:

  1. The energy values (e.g. 1,3 kwh for today and 3016,3 kwh for total). What shall I call these variables?
  2. For the pump on/off I'll just listen to 0x26A and the 9th byte for a 03 or 04 ?

@S-Przybylski
Copy link
Author

Hi proddy,
my suggestions to questions:
1.) ENERGYToday and ENERGYTotal. If you have better namings thats also fine for me!
2.) In deed, thats what i found out. I saw that protocol changes when the pump goes on/off at least 3-4 times in the past days. In the night the protocol shows always "03"... Therefore i belive that this value shows up the pump status. I never saw other values for data byte 9 (0x26A) in my configuration.

Could you solve the issue with the toggling values?
Thanks for your patience and support!

Addititional observation:
Yesterday a also saw that the 0x262 protocol updates the second value bottomtemp in a "short" protocol: 30 00 FF 02 02 62 01 B3 (CRC=BF). I rechecked the protocols and found also situations where the pump was "off" and then both bottomtemp and collectortemp was updated in a short message independently (so i have to withdraw my last statement, that the collectortemp will send out additionally in a short protocol when the pump is on).

@S-Przybylski
Copy link
Author

Hi @proody
regarding an additional protocol 0x028E:
I have identified three values in the EMS+ 0x028E Protocol which cover the Solar energy earnings - repeated every hour

  • last hour / 10 in Wh --> databyte 2+3
  • todays in Wh --> databyte 6+7
  • total / 10 in kWh --> databyte 10+11

Protocol:
(01:07:35.738) SM -> all, type 0x028E telegram: 30 00 FF 00 02 8E 00 00 0C F3 00 00 06 02 00 00 76 33 (CRC=8A), #data=11
(02:07:31.416) SM -> all, type 0x028E telegram: 30 00 FF 00 02 8E 00 00 07 9E 00 00 06 C5 00 00 76 35 (CRC=27), #data=11
(03:07:26.818) SM -> all, type 0x028E telegram: 30 00 FF 00 02 8E 00 00 00 00 00 00 06 C5 00 00 76 35 (CRC=82), #data=11

Example 1:07:

  • last hour --> 0x0CF3 = 3315 --> / 10 = 331,5Wh
  • todays --> 0x0602 = 1538Wh
  • total --> 0x7633 = 30259 --> / 10 = 3025,9 kWh

Example 2:07:

  • last hour --> 0x079E = 1950 --> / 10 = 195Wh
  • todays --> 0x06C5 = 1733Wh
  • total --> 0x7635 = 30261 --> / 10 = 3026,1 kWh

Example 3:07 (pump was of since 2:07):

  • last hour --> 0x0000 = 0000 --> / 10 = 0Wh
  • todays --> 0x06C5 = 1733Wh
  • total --> 0x7635 = 30261 --> / 10 = 3026,1 kWh

Evidence: 1733Wh - 1538Wh = 195Wh

I could also make sense to send the last hour value (the resolution of the value is higher than the total value) for individual analytic purposes. The value represents the average power of the solar panels

Does it make sense to put this information like the ones i found out up to now into a EMS+ Protocol datasheet?

Any news regarding my last post ?

@proddy
Copy link
Collaborator

proddy commented May 1, 2019

yes it would be good to add to the Wiki under the SM100 section. https://github.com/proddy/EMS-ESP/wiki/SM100

I haven't had time to look into implementing this yet, its still changing and I want to figure out how the EMS+ works for all devices.

@proddy
Copy link
Collaborator

proddy commented May 3, 2019

Added to 1.7. MQTT is like {"temp":"?","bottomtemp":"?","pump":"?","energylasthour":331.5,"energytoday":1538,"energytotal":3025.9}

@S-Przybylski
Copy link
Author

Hi @proddy
i have just tested your new release b12.
I do not get the protocol in total sync, but its close. The mqtt event toggles:
{"temp":"?","bottomtemp":43,"pumpmodulation":0,"pump":"?"}
{"temp":22.8,"bottomtemp":43,"pumpmodulation":0,"pump":"?"}
{"temp":"?","bottomtemp":43,"pumpmodulation":0,"pump":"?"}
{"temp":"?","bottomtemp":43,"pumpmodulation":0,"pump":"?"}
{"temp":21.8,"bottomtemp":43,"pumpmodulation":0,"pump":"?"}
{"temp":"?","bottomtemp":43,"pumpmodulation":0,"pump":"?"}
{"temp":"?","bottomtemp":43,"pumpmodulation":0,"pump":"?"}

(00:01:56.837) SM -> Boiler, type 0x33 telegram: 30 88 33 02 07 (CRC=C0) #data=1
(00:01:56.867) Boiler -> SM, type 0x33 telegram: 08 30 33 02 33 FB 00 1E FF 06 46 (CRC=C9) #data=7
(00:01:56.895) SM -> Boiler, type 0x35 telegram: 30 08 35 02 00 (CRC=BB) #data=1
(00:01:57.013) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 00 E7 01 AE 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=C9) #data=24
<--- SM100Monitor(0x262)
(00:01:57.390) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)
(00:01:57.611) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:01:58.150) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 00 05 64 00 00 00 00 (CRC=81) #data=17
<--- SM100Status(0x264)
(00:01:58.364) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:01:58.800) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:01:59.024) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5) #data=12
<--- SM100Status2(0x26A)
Publishing SM data via MQTT

(00:02:26.955) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:02:46.920) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 00 E4 (CRC=CA) #data=2
<--- SM100Monitor(0x262)
Publishing SM data via MQTT
(00:02:56.862) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 00 E3 01 AE 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=9E) #data=24
<--- SM100Monitor(0x262)
(00:02:57.275) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)
Publishing SM data via MQTT
(00:02:57.493) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:02:57.989) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 00 05 64 00 00 00 00 (CRC=81) #data=17
<--- SM100Status(0x264)
(00:02:58.195) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:02:58.578) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:02:58.811) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5) #data=12
<--- SM100Status2(0x26A)

Requesting type SM10Monitor(0x97) from dest 0x30

(00:03:10.628) 0x09 -> SM, type 0x02 telegram: 09 B0 02 00 03 (CRC=66) #data=1
(00:03:10.655) SM -> 0x09, type 0x02 telegram: 30 09 02 00 A3 15 04 (CRC=25) #data=3
(00:03:10.667) 0x09 -> SM, type 0x96 telegram: 09 B0 96 00 07 (CRC=00) #data=1
(00:03:10.715) SM -> 0x09, type 0x96 telegram: 30 09 96 00 FF 18 1E 0A 02 50 28 (CRC=B5) #data=7
(00:03:10.736) 0x09 -> SM, type 0xFF00 telegram: 09 B0 FF 00 FF 00 01 (CRC=0C) #data=1
(00:03:10.775) SM -> 0x09, type 0x0001 telegram: 30 09 FF 00 00 01 (CRC=70)

(00:03:11.168) Sending read of type 0x97 to 0x30: telegram: 0B B0 97 00 20 (CRC=03)
(00:03:11.202) SM -> me, type 0x97 telegram: 30 0B 97 00 (CRC=82)

(00:03:23.421) 0x09 -> SM, type 0x02 telegram: 09 B0 02 00 03 (CRC=66) #data=1
(00:03:23.456) SM -> 0x09, type 0x02 telegram: 30 09 02 00 A3 15 04 (CRC=25) #data=3
(00:03:23.474) 0x09 -> SM, type 0x96 telegram: 09 B0 96 00 07 (CRC=00) #data=1
(00:03:23.507) SM -> 0x09, type 0x96 telegram: 30 09 96 00 FF 18 1E 0A 02 50 28 (CRC=B5) #data=7
(00:03:23.524) 0x09 -> SM, type 0xFF00 telegram: 09 B0 FF 00 FF 00 01 (CRC=0C) #data=1
(00:03:23.556) SM -> 0x09, type 0x0001 telegram: 30 09 FF 00 00 01 (CRC=70)

(00:03:26.036) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4

(00:03:56.094) SM -> Boiler, type 0x33 telegram: 30 88 33 02 07 (CRC=C0) #data=1
(00:03:56.146) SM -> Boiler, type 0x35 telegram: 30 08 35 02 00 (CRC=BB) #data=1
(00:03:56.267) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 00 E0 01 AE 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=E6) #data=24
<--- SM100Monitor(0x262)
(00:03:56.698) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)
(00:03:56.930) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:03:57.375) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 00 05 64 00 00 00 00 (CRC=81) #data=17
<--- SM100Status(0x264)
(00:03:57.579) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:03:58.038) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:03:58.264) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5) #data=12
<--- SM100Status2(0x26A)
Publishing SM data via MQTT

(00:04:11.586) Sending read of type 0x97 to 0x30: telegram: 0B B0 97 00 20 (CRC=03)
(00:04:11.648) SM -> me, type 0x97 telegram: 30 0B 97 00 (CRC=82)
(00:04:26.196) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4

(00:04:56.383) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 00 DC 01 AE 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=D0) #data=24
<--- SM100Monitor(0x262)
(00:04:56.843) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)
(00:04:57.082) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:04:57.528) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 00 05 64 00 00 00 00 (CRC=81) #data=17
<--- SM100Status(0x264)
(00:04:57.735) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:04:58.185) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:04:58.400) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5) #data=12
<--- SM100Status2(0x26A)

For me it seems that the pump doesn't function, the collectors temp (here it began to rain, therefore it often updates the 0x262 protocol (long and short) to reflect the falling temperatures.

Do you have an Idea, why the "?" and "0" comes up?
I do not get the above protocol. Its still the old one. The topic switched to "sm_data"

@proddy
Copy link
Collaborator

proddy commented May 3, 2019

I will supress the ? values and check why the pump on/off is not working. It works with the previous test date you provided. The pump modulation is missing for SM100 as I don't know where it is stored.

@proddy
Copy link
Collaborator

proddy commented May 3, 2019

made a fix in b13. I had to change some of the MQTT topic names as there are multiple solar modules (sm10, sm100, junkers, nefit etc)

pump modulation is in, i forgot. "test 15" shows it working based on your research,

@S-Przybylski
Copy link
Author

Hi @proddy
here by my test with the Version b13. I put the mqtt events and comments into it (shortend):

Device found. Model SM100 Solar Module with DeviceID 0x30, ProductID 163, Version 21.04
SM10 Solar Module support enabled.
...
Publishing SM data via MQTT {"bottomtemp":40.6,"pumpmodulation":0}

(00:02:11.146) Sending read of type 0x97 to 0x30: telegram: 0B B0 97 00 20 (CRC=03)
(00:02:11.214) SM -> me, type 0x97 telegram: 30 0B 97 00 (CRC=82)
(00:02:33.376) SM -> Boiler, type 0x33 telegram: 30 88 33 02 07 (CRC=C0) #data=1
(00:02:33.402) Boiler -> SM, type 0x33 telegram: 08 30 33 02 33 FB 00 1E FF 06 46 (CRC=C9) #data=7
(00:02:33.432) SM -> Boiler, type 0x35 telegram: 30 08 35 02 00 (CRC=BB) #data=1
(00:02:33.542) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 00 3A 01 96 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=E3) #data=24
<--- SM100Monitor(0x262)

  • Publishing SM data via MQTT {"temp":5.8,"bottomtemp":40.6,"pumpmodulation":0}
    #Comment: temp and bottomtemp correct! pumpmodulation from log file perspective unknown

(00:02:33.961) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)

  • Publishing SM data via MQTT {"bottomtemp":40.6,"pumpmodulation":0}
    #Comment: no need to repeat value; no new value or update given; missing temp

(00:02:34.186) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:02:34.697) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 00 00 00 FF 00 00 00 00 05 64 00 00 00 00 (CRC=AD) #data=17
<--- SM100Status(0x264)
(00:02:34.900) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:02:35.335) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:02:35.554) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5) #data=12
<--- SM100Status2(0x26A)
(00:03:03.573) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
Requesting type SM10Monitor(0x97) from dest 0x30
(00:03:12.572) Sending read of type 0x97 to 0x30: telegram: 0B B0 97 00 20 (CRC=03)
(00:03:12.629) SM -> me, type 0x97 telegram: 30 0B 97 00 (CRC=82)
(00:03:33.674) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 00 3A 01 96 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=E3) #data=24
<--- SM100Monitor(0x262)

  • Publishing SM data via MQTT {"temp":5.8,"bottomtemp":40.6,"pumpmodulation":0}
    #Comment: all three values ok; missing pump (refer to 0x26A)

(00:03:34.173) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)

  • Publishing SM data via MQTT {"bottomtemp":40.6,"pumpmodulation":0}
    #Comment: no need to repeat value; no new value or update given; missing temp and pump (both from older values)

(00:03:34.393) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:03:34.863) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 00 00 00 FF 00 00 00 00 05 64 00 00 00 00 (CRC=AD) #data=17
<--- SM100Status(0x264)
(00:03:35.067) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:03:35.471) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:03:35.685) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5) #data=12
<--- SM100Status2(0x26A)
(00:04:02.808) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4

  • Publishing SM data via MQTT {"bottomtemp":40.6,"pumpmodulation":0}
    #Comment: missing temp (from older value) and pump (refer to 0x26A)

What is the operational mode to store the solar values?

  1. Do you store all known values and updates these accordingly? or
  2. Do you store some values and others are volatile?
    For my it seems that the program currently act as described in 2.
    I would prefer 1.
    Benefit: All values are send to HomeAssistant even if some of them are not updated. And you don't get update error in the HA log file (while mode 1. produces errors)...

@proddy
Copy link
Collaborator

proddy commented May 3, 2019

thanks for checking. I'll use your telegrams and simulate the behaviour here so I can debug.

It is #1, all values are stored and updated but sometimes I've noticed you get partial telegrams so values are overwritten sometimes incorrectly. I'll fix that.

@proddy
Copy link
Collaborator

proddy commented May 4, 2019

made the changes. please test

@S-Przybylski
Copy link
Author

S-Przybylski commented May 4, 2019

Hi @proddy
thanks for the fast update! I have tested the todays version. For me it seems that three things are not working correctly:

  1. pumpmodulation isn't set up correctly - the short message was ignored
    SM -> all, type 0x0264 telegram: 30 00 FF 09 02 64 64 (CRC=37) #data=1
  2. pump status isn't set up correctly - the long message was wrongly interpreted; 04 is on while 03 is off
    SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 04 03 (CRC=EB) #data=12
  3. Strange thing: The last mqtt messages show negative values for "bottomtemp" while no SM-update occurred. Do you have an idea why this value is overwritten by an other process?

Commented Logfile incl. mqtt messages:
...
(00:01:38.719) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
Publishing SM data via MQTT
Requesting type SM10Monitor(0x97) from dest 0x30
(00:02:07.941) SM -> Boiler, type 0x33 telegram: 30 88 33 02 07 (CRC=C0) #data=1
(00:02:07.972) Boiler -> SM, type 0x33 telegram: 08 30 33 02 33 FB 00 1E FF 06 46 (CRC=C9) #data=7
(00:02:07.999) SM -> Boiler, type 0x35 telegram: 30 08 35 02 00 (CRC=BB) #data=1
(00:02:08.117) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 01 AB 01 4E 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=FB) #data=24
<--- SM100Monitor(0x262)
Publishing SM data via MQTT (not taken)
(00:02:08.971) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)
(00:02:09.187) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:02:10.032) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 00 05 64 00 00 00 00 (CRC=81) #data=17
<--- SM100Status(0x264)
(00:02:10.240) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:02:10.849) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:02:11.035) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5) #data=12
(00:02:12.055) SM -> me, type 0x97 telegram: 30 0B 97 00 (CRC=82)
(00:02:14.938) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 01 AC (CRC=80) #data=2
<--- SM100Monitor(0x262)

Publishing SM data via MQTT {"collectortemp":42.8,"pumpmodulation":0,"pump":"off"}
#Comment: Status seems to be ok; missing protocol details

(00:02:38.974) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:02:41.949) SM -> all, type 0xBF telegram: 30 00 BF 00 30 A3 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 (CRC=45) #data=24
(00:03:08.120) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 01 B1 01 4E 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=B7) #data=24
<--- SM100Monitor(0x262)

Publishing SM data via MQTT {"collectortemp":43.3,"bottomtemp":33.4,"pumpmodulation":0,"pump":"off"}
#Comment: OK

(00:03:08.725) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)
(00:03:08.949) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:03:09.487) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 00 00 05 64 00 00 00 00 (CRC=81) #data=17
<--- SM100Status(0x264)
(00:03:09.699) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:03:12.485) SM -> me, type 0x97 telegram: 30 0B 97 00 (CRC=82)
(00:03:19.860) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:03:20.077) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 03 03 (CRC=E5) #data=12
<--- SM100Status2(0x26A)
(00:03:20.834) 0x09 -> SM, type 0x02 telegram: 09 B0 02 00 03 (CRC=66) #data=1
(00:03:20.866) SM -> 0x09, type 0x02 telegram: 30 09 02 00 A3 15 04 (CRC=25) #data=3
(00:03:20.878) 0x09 -> SM, type 0x96 telegram: 09 B0 96 00 07 (CRC=00) #data=1
(00:03:20.926) SM -> 0x09, type 0x96 telegram: 30 09 96 00 FF 18 1E 0A 02 50 28 (CRC=B5) #data=7
(00:03:20.948) 0x09 -> SM, type 0xFF00 telegram: 09 B0 FF 00 FF 00 01 (CRC=0C) #data=1
(00:03:20.986) SM -> 0x09, type 0x0001 telegram: 30 09 FF 00 00 01 (CRC=70)
(00:03:28.890) SM -> all, type 0x0264 telegram: 30 00 FF 09 02 64 64 (CRC=37) #data=1
<--- SM100Status(0x264)
(00:03:29.346) SM -> all, type 0x026A telegram: 30 00 FF 0A 02 6A 04 (CRC=53) #data=1
<--- SM100Status2(0x26A)

Publishing SM data via MQTT {"collectortemp":43.3,"bottomtemp":33.4,"pumpmodulation":0,"pump":"on"}
#Comment: pump goes to on - ok; wrong - pumpmodulation goes to "0x64" = 100% - message 0x264 offset 09 0x64

(00:03:32.908) SM -> all, type 0x0264 telegram: 30 00 FF 09 02 64 1E (CRC=4D) #data=1
<--- SM100Status(0x264)
(00:03:38.049) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4

Publishing SM data via MQTT {"collectortemp":43.3,"bottomtemp":33.4,"pumpmodulation":0,"pump":"on"}
#Comment: wrong - pumpmodulation switched to 1E = 30% - message 0x264 offset 09 0x1E

(00:04:07.258) SM -> Boiler, type 0x33 telegram: 30 88 33 02 07 (CRC=C0) #data=1
(00:04:07.292) Boiler -> SM, type 0x33 telegram: 08 30 33 02 33 FB 00 1E FF 06 46 (CRC=C9) #data=7
(00:04:07.325) SM -> Boiler, type 0x35 telegram: 30 08 35 02 00 (CRC=BB) #data=1
(00:04:07.430) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 01 B1 01 4E 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 80 00 (CRC=B7) #data=24
<--- SM100Monitor(0x262)
(00:04:08.108) SM -> all, type 0x0262 telegram: 30 00 FF 18 02 62 80 00 (CRC=AE) #data=2
<--- SM100Monitor(0x262)
(00:04:08.339) SM -> all, type 0x0263 telegram: 30 00 FF 00 02 63 80 00 80 00 00 00 80 00 80 00 80 00 00 (CRC=74) #data=13
(00:04:08.804) SM -> all, type 0x0264 telegram: 30 00 FF 00 02 64 00 00 00 04 00 00 FF 00 00 1E 00 05 64 00 00 00 00 (CRC=06) #data=17
<--- SM100Status(0x264)

Publishing SM data via MQTT {"collectortemp":43.3,"bottomtemp":33.4,"pumpmodulation":30,"pump":"on"}
#Comment: now status ok; the full 0x0264 protocol seems to be interpreted correctly

(00:04:09.008) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:04:09.412) SM -> all, type 0x0268 telegram: 30 00 FF 00 02 68 0C 00 (CRC=1E) #data=2
(00:04:09.626) SM -> all, type 0x026A telegram: 30 00 FF 00 02 6A 03 03 03 00 03 03 03 03 03 00 04 03 (CRC=EB) #data=12
<--- SM100Status2(0x26A)

Publishing SM data via MQTT {"collectortemp":43.3,"bottomtemp":33.4,"pumpmodulation":30,"pump":"off"}
#Comment: pump status wrong - 0x026A databyte 0A reports (again) x04 - that means pump is and here stays on (status x03 is off)

(00:04:30.820) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 01 A1 (CRC=8D) #data=2
<--- SM100Monitor(0x262)

Publishing SM data via MQTT {"collectortemp":41.7,"bottomtemp":-2944,"pumpmodulation":30,"pump":"off"}
#Comment: Collectortemp update ok! WRONG - Why bottomtemp says suddenly "-2944" ????????????????????????????

(00:04:38.164) SM -> all, type 0x0266 telegram: 30 00 FF 00 02 66 01 62 00 12 (CRC=73) #data=4
(00:04:46.709) SM -> all, type 0x0262 telegram: 30 00 FF 00 02 62 01 95 (CRC=B9) #data=2
<--- SM100Monitor(0x262)

Publishing SM data via MQTT {"collectortemp":40.5,"bottomtemp":-1817.6,"pumpmodulation":30,"pump":"off"}
#Comment: Collectortemp update ok! WRONG - Why is bottomtemp says now "-1817.6" ????????????????????????????

@proddy
Copy link
Collaborator

proddy commented Jun 9, 2019

@DarthMob it's probably an issue in how the JSON is rendered. Could you add a line in ems-esp.cpp to print out the real value before it's sent via MQTT. So somewhere around line #765 after the if (EMS_Other.SM) {

add myDebug("!! value = %d", EMS_Other.SMEnergyTotal);

then upload, wait a while, do a publish and see what the value is. Hopefully its not negative!

@DarthMob
Copy link

DarthMob commented Jun 16, 2019

@proddy
thanks for the info.
I tried doing that with version 1.8 (had to update PIO to v4) but now i have a new issue:
-> as soon as i exit serial mode and ems-esp tries to read the EMS bus i get frequent reboots of the system!

Setup with bbqkees board is exactly the same!
I tried everything (erasing flash, compiling without the debug line, fresh 1.8 download...)

as you can see here from the PING there is no stable connection when reading from EMS.
After "set serial on" everything is fine (done in the middle of the picture)

image

version 1.7 worked fine (except the issue with "-"kWh)..
any ideas?

@proddy
Copy link
Collaborator

proddy commented Jun 16, 2019

@DarthMob that's really strange indeed and can't think what is causing this. If you can't get to the console then you won't be able to see the stack dump to debug the cause or even use system to see whether its an ESP reset or a wifi disruption. Does it happen immediately after restarting? Or during the telnet session? Also I assume this is Wemos D1 mini with 4MB flash?

Some things you could try

  1. Using platform = ${common.arduino_core_2_4_2} in the ini file and rebuilding
  2. Grab 1.7.0 which you know works (bottom left of PlatformIO window click and select the 1.7.0 tag and rebuild after updating the libraries

@susisstrolch
Copy link

susisstrolch commented Jun 16, 2019 via email

@proddy
Copy link
Collaborator

proddy commented Jun 16, 2019

that's not good. There are some reports of people with WiFi issues since the 2.5.0 core, maybe its related?
esp8266/Arduino#5908
esp8266/Arduino#2795

@susisstrolch
Copy link

Here‘s an excerpt from my mqtt server.
ˋˋˋ
setstate MQTT2_ems_esp uptime: 63s
setstate MQTT2_ems_esp 2019-06-16 16:21:18 ServiceCode 0H
setstate MQTT2_ems_esp 2019-06-16 16:21:18 ServiceCodeNumber 0
setstate MQTT2_ems_esp 2019-06-16 16:21:18 boilTemp 20.4
setstate MQTT2_ems_esp 2019-06-16 16:21:18 burnGas off
setstate MQTT2_ems_esp 2019-06-16 16:21:18 burnStarts 138298
setstate MQTT2_ems_esp 2019-06-16 16:21:18 burnWorkMin 520179
setstate MQTT2_ems_esp 2019-06-16 16:21:18 curBurnPow 0
setstate MQTT2_ems_esp 2019-06-16 16:21:18 curFlowTemp 20.4
setstate MQTT2_ems_esp 2019-06-16 16:21:18 fanWork off
setstate MQTT2_ems_esp 2019-06-16 16:21:18 heatPmp off
setstate MQTT2_ems_esp 2019-06-16 16:21:18 heatWorkMin 518615
setstate MQTT2_ems_esp 2019-06-16 16:21:18 heating_active 0
setstate MQTT2_ems_esp 2019-06-16 16:21:18 ignWork off
setstate MQTT2_ems_esp 2019-06-16 16:21:18 outdoorTemp 23.9
setstate MQTT2_ems_esp 2019-06-16 16:21:18 pumpMod 0
setstate MQTT2_ems_esp 2019-06-16 16:21:18 selBurnPow 100
setstate MQTT2_ems_esp 2019-06-16 16:21:18 selFlowTemp 11
setstate MQTT2_ems_esp 2019-06-16 18:28:52 shower_alert 0
setstate MQTT2_ems_esp 2019-06-16 18:28:52 shower_timer 0
setstate MQTT2_ems_esp 2019-06-16 18:28:52 start start
setstate MQTT2_ems_esp 2019-06-16 18:28:52 status online
setstate MQTT2_ems_esp 2019-06-15 13:22:27 tapwater_active 0
setstate MQTT2_ems_esp 2019-06-16 16:21:18 uptime 63
setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWActivated off
setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWCirc off
setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWComfort Hot
setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWHeat off
setstate MQTT2_ems_esp 2019-06-16 16:21:18 wWSelTemp 60

ˋˋˋ
As you cam See it mostly stalls after sending the shower info...
Maybe connection problems with the mqtt Server?

@DarthMob
Copy link

DarthMob commented Jun 16, 2019

i will try, however i have absolutely no wifi issue while serial console is "on" (i'm connected trough wifi+telnet !!).
only after ems-esp goes into reading the EMS bus (serial mode off) the connection loss appears

PS: i have a nodemcuv2 board

@proddy
Copy link
Collaborator

proddy commented Jun 17, 2019

@DarthMob ok, one other thing to try is set listen_mode on which will disable all Tx. You should still see data coming in (make sure serial is off). If it works then it means the new Tx code is causing the crash. I'll work on extra debugging tonight

@proddy
Copy link
Collaborator

proddy commented Jun 17, 2019

@susisstrolch could you also try with set listen_mode on to see if your device stays on longer than 1500s. This disables the Tx logic. Also if you do a set publish_time 0 this will disable the MQTT posts. I'm really trying to isolate what is causing the crashes!

@d-a-v
Copy link

d-a-v commented Jun 17, 2019

You can enable software serial to enable logs on some other pins while hardware serial is in use.
You can also move hardware serial to other pins and still use regular pins with software serial.
You won't get core logs with software serial, but you can still enable yours.
Check this
Don't use core 2.5.0, try 2.5.2 instead.
(Sorry for intrusion)

@DarthMob
Copy link

@proddy
it works! with 1.8 and listen_mode "on" everything is working - so Tx seems to be the problem?

i also used the "myDebug("!! value = %d", EMS_Other.SMEnergyTotal);" as you requested and after publish i get the following value

publish
!! value = -32768

:-(

@proddy
Copy link
Collaborator

proddy commented Jun 19, 2019

@DarthMob ok so there is an endless loop in the new Tx code that causes the WDT resets. I'll have to debug this with @susisstrolch. Strange this I don't experience this on my setup.

The SMEnergyTotal is captured via sending a request, so since Tx is disabled (with the listen_mode) it'll always be a negative value. I'll make a quick modification to suppress it.

@susisstrolch
Copy link

I see something completely different - no endless loop, but

[CRASH] Last crash was 0 days 0 hours 0 minutes 8 seconds since boot time
[CRASH] Reason of restart: 2 - Fatal exception
[CRASH] Exception cause: 9 - LoadStoreAlignmentCause
[CRASH] epc1=0x40223e72 epc2=0x00000000 epc3=0x00000000
[CRASH] excvaddr=0x7e032096 depc=0x00000000
[CRASH] sp=0x3ffffe90 end=0x3fffffc0
>>>stack>>>

clean build from current master, no personal modifications, running on ESP8266-12F

@susisstrolch
Copy link

cont... which translates to

stack:
0x40224020: sflags(char const*, fs::OpenMode&, fs::AccessMode&) at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/FS.cpp:390
0x3fff4a5c: ?? at /home/gauchard/dev/esp8266/esp8266/tools/sdk/lwip2/builder/glue-lwip/esp-dhcpserver.c:31
0x402113c8: uart_set_debug at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/uart.cpp:900
0x402117f0: malloc at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/umm_malloc/umm_malloc.cpp:1685
0x4022d20a: strstr at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/string/strstr.c:80
0x40209737: MyESP::myDebug_P(char const*, ...) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:134
0x4022d1f8: strstr at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/string/strstr.c:69
0x40246fb8: umm_info at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/umm_malloc/umm_malloc.cpp:1088
0x4020fb00: pinMode at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_wiring_digital.cpp:60 (discriminator 4)
0x4024f9d5: system_station_got_ip_set at ??:?
0x4020d0d0: MyESP::begin(char const*, char const*, char const*) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:1803
0x4020aa5c: MyESP::showSystemStats() at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:1154 (discriminator 9)
0x3ffe8304: ?? at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/reent/impure.c:23
0x40227977: strcasecmp at /home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/string/strcasecmp.c:54
0x4020c9e1: MyESP::_changeSetting(unsigned char, char const*, char const*) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:721
0x4021813e: ClientContext::_write_from_source(DataSource*) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266WiFi/src/include/ClientContext.h:445
0x4020cfbd: MyESP::_wifiCallback(justwifi_messages_t, char*) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:210
0x3fff4a1c: ?? at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/StackThunk.cpp:34
0x4020d090: MyESP::begin(char const*, char const*, char const*) at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:1790
0x40205cbc: _GLOBAL__sub_I_ds18 at ~/Development/esp8266/EMS-ESP-Boiler/src/ems-esp.cpp:19
  \-> inlined by: _GLOBAL__sub_I_ds18 at ~/Development/esp8266/EMS-ESP-Boiler/src/ems-esp.cpp:1745
0x4020f088: crc_update$constprop$0 at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_eboot_command.cpp:28
0x401006f1: cont_wrapper at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/cont.S:82

Looks like arduino library...
(btw, I'm using your latest platformio.ini-example)...

@proddy
Copy link
Collaborator

proddy commented Jun 19, 2019

@susisstrolch It's hard to see from the stack what caused what - I've always found it slightly misleading. Did this crash occur after a telnet command (like a setting change or a 'system')? Could it be the Telnet client like last time? Can you reproduce it also via the Serial? Running in a serial monitor will also dump the stack. You could also try 'crash test 1' which causes a crash and then compare whether the stack is pointing to the right code. For some reason I just don't trust it 100%. Anyways. need something reproducible so I can home in on the root cause. I've been running 1.8 for 4 days no with no outages.

@susisstrolch
Copy link

I've connected the Tx/Rx (GPIO1, GPIO3) to an ESPlink adapter for monitoring purposes. But I don't see the crash dump.
Also, master doesn't have the 'crash test 1' command...

@susisstrolch
Copy link

argh... the stackdump file is hardcoded in the script?

@proddy
Copy link
Collaborator

proddy commented Jun 19, 2019

crash test was added back to to the dev. How I use the stackdump is by copying the stack <<</>>> section and pasting into the file stackdmp.txt. Then from my PlatformIO right-click on the analyze_stackdmp.py file and hit run.

@susisstrolch
Copy link

Jep... but that won't work on linux because of two flaws
a) there's nothing like .exe
b) scripts/analyze_stackdmp.py has a trailing space in calling decode.py

So, here's the latest coredump. For me it seems to be an issue with the runtime (especially mDNS):

stack:
0x40223128: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_A(IPAddress, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:1387
0x40208de4: OneWire::write(unsigned char, unsigned char) at ~/Development/esp8266/EMS-ESP-Boiler/.pio/libdeps/debug/OneWire_ID1/OneWire.cpp:259
0x40223180: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_PTR_IP4(IPAddress, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:1408
0x40220000: esp8266::MDNSImplementation::MDNSResponder::_updateProbeStatus() at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/PolledTimeout.h:185
  \-> inlined by: esp8266::MDNSImplementation::MDNSResponder::_updateProbeStatus() at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Control.cpp:1117
0x40223e6a: ArduinoJson6110_00000::JsonDeserializer<ArduinoJson6110_00000::ArduinoStreamReader, ArduinoJson6110_00000::StringCopier>::current() at ~/Development/esp8266/EMS-ESP-Boiler/.pio/libdeps/debug/ArduinoJson_ID64/src/ArduinoJson/Json/JsonDeserializer.hpp:51
0x402232c5: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_PTR_TYPE(esp8266::MDNSImplementation::MDNSResponder::stcMDNSService&, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:1451
0x40223355: esp8266::MDNSImplementation::MDNSResponder::_writeMDNSAnswer_TXT(esp8266::MDNSImplementation::MDNSResponder::stcMDNSService&, esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:1490
0x4024760c: ppPeocessRxPktHdr at ??:?
0x40210250: SPIFFSFileImpl::size() const at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/spiffs_api.h:408
0x40250029: wifi_set_broadcast_if at ??:?
0x4020d794: fs::File::peek() at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/FS.cpp:76
0x4020ad05: MyESP::_consoleShowHelp() at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/WString.h:272
  \-> inlined by: ?? at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/WString.h:211
  \-> inlined by: MyESP::_consoleShowHelp() at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:530
0x40223e93: ArduinoJson6110_00000::JsonDeserializer<ArduinoJson6110_00000::ArduinoStreamReader, ArduinoJson6110_00000::StringCopier>::canBeInNonQuotedString(char) at ~/Development/esp8266/EMS-ESP-Boiler/.pio/libdeps/debug/ArduinoJson_ID64/src/ArduinoJson/Json/JsonDeserializer.hpp:313 (discriminator 3)
0x40225578: esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter::findCachedDomainOffset(void const*, bool) const at ??:?
0x40225578: esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter::findCachedDomainOffset(void const*, bool) const at ??:?
0x40225578: esp8266::MDNSImplementation::MDNSResponder::stcMDNSSendParameter::findCachedDomainOffset(void const*, bool) const at ??:?
0x401006cd: cont_continue at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/cont.S:51
0x40223f0f: EspClass::magicFlashChipMode(unsigned char) at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/Esp.cpp:359
0x402228b0: esp8266::MDNSImplementation::MDNSResponder::_readRRDomain_Loop(esp8266::MDNSImplementation::MDNSResponder::stcMDNS_RRDomain&, unsigned char) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Transfer.cpp:809
0x4021fc9c: esp8266::MDNSImplementation::MDNSResponder::_parseQuery(esp8266::MDNSImplementation::MDNSResponder::stcMDNS_MsgHeader const&) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Control.cpp:376 (discriminator 1)
0x402205e2: esp8266::MDNSImplementation::MDNSResponder::_processAnswers(esp8266::MDNSImplementation::MDNSResponder::stcMDNS_RRAnswer const*) at ~/.platformio/packages/framework-arduinoespressif8266/libraries/ESP8266mDNS/src/LEAmDNS_Control.cpp:715
0x40100326: millis at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_wiring.cpp:186
0x4020ae6c: MyESP::_consoleShowHelp() at ~/Development/esp8266/EMS-ESP-Boiler/lib/MyESP/MyESP.cpp:566
0x3fff2a7c: ?? at ~/Development/esp8266/EMS-ESP-Boiler/src/ems-esp.cpp:63
0x4020d675: EspClass::getFreeSketchSpace() at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/Esp.cpp:535
0x3fff2a7c: ?? at ~/Development/esp8266/EMS-ESP-Boiler/src/ems-esp.cpp:63
0x40205d90: _process_UBAParametersMessage(_EMS_RxTelegram*) at ~/Development/esp8266/EMS-ESP-Boiler/src/ems.cpp:1104
0x4020f7d8: init at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_wiring.cpp:214
0x3ffe9720: ?? at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/core_esp8266_main.cpp:55
0x401006e9: cont_wrapper at ~/.platformio/packages/framework-arduinoespressif8266/cores/esp8266/cont.S:81

@proddy
Copy link
Collaborator

proddy commented Jun 19, 2019

it was a simple fix to get it working with linux
python scripts/decoder_linux.py -s -e .pio/build/debug/firmware_d1_mini.elf scripts/stackdmp.txt

@susisstrolch
Copy link

@proddy I've pushed some changes to https://github.com/susisstrolch/EMS-ESP/tree/dev, trying to fix reboot problem. Major change:

  • emsuart.cpp
    check for EMSbus break during send
    add a loopcount to avoid endless wait if send char doesn't get echoed
    emsuart_tx_buffer returns a 32bit result, holding Tx result, BRK and watchdog

  • debug output for ems.cpp
    show 'poll us'
    enable the 'answer poll request'

When sending to EMSbus I see a EMSbus BRK approx. 10 bittimes after sending the 3rd byte to the EMSbus:

ems_parseTelegram: poll us
ems_parseTelegram: emsuart_tx_poll()
ems_parseTelegram: poll us
ems_parseTelegram: emsuart_tx_poll()
(02:27:19.731) 08 00 18 00 0B 00 D3 64 00 00 00 00 00 83 00 7D 00 80 00 00 00 FF 30 48 00 00 FF 00 00 83 00 54
(02:27:20.012) 08 00 18 1B 00 00 00 00 00 00 00 0B 00 00 FF
ems_parseTelegram: poll us
(02:27:20.944) Sending read of type 0x16 to 0x08: telegram: 0B 88 16 00 20 (CRC=EC) #data=224
_ems_sendTelegram: BRK from EMS Busmaster [3,10]!
ems_parseTelegram: poll us

Because I can't connect the logic analyzer it's hard for me to see what really happens.
Maybe my hardware is faulty...
So, can you try my dev build to see if the same happens on your hardware?

@proddy
Copy link
Collaborator

proddy commented Jun 21, 2019

nice, I'll check it out tonight and hook up the logic analyzer. Noticed the #data=224. The buffer is only 32 bytes so this may be causing the pointer/memory exceptions. Perhaps but a test in emsuart to only add to the buffer if the length <= 32.

@susisstrolch
Copy link

Hmm, not sure where it gets the #data=224 from..
The 'BRK from EMS Busmaster [3,10]' means break while sending 3rd byte, 10 bit-times after feeding the fifo. At this point we only have send src, dst and type.

@proddy
Copy link
Collaborator

proddy commented Jun 21, 2019

@susisstrolch testing your dev build and I'm not seen any errors like you get. Seems to work fine, but then again version 1.7 and 1.8 are also stable with me. Maybe it's the wifi speed? Does changing EMSESP_DELAY in ems-esp.cpp make a difference perhaps?

Capture

@susisstrolch
Copy link

susisstrolch commented Jun 21, 2019 via email

@S-Przybylski
Copy link
Author

dear all
i want to close the issue. Please set up a new dedicated issue for existing problems instead using this issue (like i have done by now).
Thanks

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants