Help making Grott more user friendly

[johanmeijer]

Last days Growatt servers seems to be very unstable. This resulted in a lot more visitors and download from the Grott github repository.

Searching on the Internet about reactions on and information about the use of grott I see some reaction that Grott is to difficult to use.

Of course I want to make grott more user friendly (I am a techie and for some things are obvious). So please can you help (hints and tips, advise) how to improve grott/grottserver (and documentation).

[Perrylicious]

There was a long text here but I conflated some stuff on your wiki with guides written by muppet3000, will rewrite tomorrow

[GeraldPape]

For me it was difficult to get started. (Using the grott edge ha plugin).
I missed the inverter type setting as it was not mentioned in the description.

[johanmeijer]

I will make that more clear (my intention is to eliminate this in a next release, make grott more auto detect the inverter type).

[JazekerXX]

Hallo Johan, thx for your work on this HA integration! it would be great if we could install this with hacs or the HA integration page. i'm currently a user of the Growatt integration available in the HA integration page, but i get insane by the continuously stopping of the integration. i know it's because of the servers in china but it's really annoying having to wait for a HA update every time. so that's why i like youre approach! i'm not shure how to install this on a non docker HA version. so i think the solution of making this more user friendly is to making it available in hacs or the integration page.

[bardahlm]

At the moment using Grott adds too many moving parts. You need to keep both Grott and an MQTT broker running for it to work, in addition to whatever is consuming the data (Home Assistant in my usecase). So in a perfect world one could install Grott as an integration in HA and have Home Assistant run the service and make sure it is running and exposing an listening port that the inverters can talk to. All metrics will be native Home Assistant entities. If one need the data other places (Influx or MQTT) one can use the existing HA functions to export data.

[johanmeijer]

Hmm making a HA only version does not have my priority. I created grott for my own (home-automation) environment running MQTT (with mainly nodered as listener), influxdb with Grafana and sending data to PVOutoput.

I have not HA running, maybe I am going to do this in the future. Is at my todo list, but to be honest Grott consumes a lot of my time and does not leave much room for other projects at this moment.

[typxxi]

I created grott for my own (home-automation) environment running MQTT (with mainly nodered as listener), influxdb with Grafana and sending data to PVOutoput.

I am a home assistant user too and even though there are hickups like the growatt killed my grott server entry in all 8 growatt inverters I am now quite familiar with it over the past 3 months or rather 4 to 5 months. Sorry, can't remember. Most of my inverters are grid tied only and only 1 is a hybrid SPH 4600 just to make things clear.
Growatt inverters usually do not take much time to maintain except the hybrid which is a completely different picture.

I totally agree that you follow your main purpose and ideas where HA support might be a spin off or a side project maintained by others.
I know about the complexity and that a lot has to be running properly from HA over mqqt and so on but that is something I can live with cause it is based on clean standards and integrated quite open.

I do not need an out of the box installation.

What i suggest to happen soon is a clean structure regarding those side or spin off projects especially regarding the "naming".

I assume we will see sooner or later GROTT 3.0
and then I hope to see a cleaner HA support by
GROTT 3.0 - HA Edition (aka the egguy which is the Grott HA addon)
GROTT 3.0 - SENSOR MANAGEMENT (which is the integration muppet3000 has developed but got a different name and I guess also egguys sensor or plugin)
GROTT 3.0 - INVERTER MANAGEMENT (which muppet 3000 is been working on to write values into the inverter registers)

Especially the eco system for HA has to be more clean and simple and stringent cause for newbies it "feels" complicated or overwhelming how many and what steps have to be done. At the end it was quite easy with the support I have gotten from egguy and muppet3000 but I had about 1,5 years before that installation where I started reading to prepare cause our 8 inverters did not arrive over night but piece by piece and the first from 2018 has been replaced and sold already.

And if the bits and pieces have clear name we will no longer the confusion of the whole terminologies.

For example you have developed a 2.8.3 which was only available as beta for a long time since it became a stable release in may 2024 (but again I am not really sure about if that is correct cause i have not really checked the grott page that much.

So I believed and thought for whatever reason that the the HA Add on Grott server would be 2.8.3 too, but I am not sure and I had to start digging deeper in the last week, when I was hit a bit later by the growatt deletion of my grottserve in the wifi sticks.

This journey starts here in ha:

The add on title is "Grott Beta branch (2.8)"
I do not like that beta has become part of a name cause it makes me and other newbies hesitant if I should start it this way.
But I understand somehow that it should refer to the genuine grott server which was also back then beta or so.

Then there is the real and for me important add on version 0.1.7 and a change log I really like. Then there is also a link to Grott Beta branch (2.8).
Please open the link and you will end up on a page which is called "addon-grott-beta" and not Grott beta branch (2.8) as before.
Really confusing if you had read before about grott, grott add on and grott beta branch 2.8.

And if you click inside the HA addon on the documentation page you will get again not grott beta branch 2.8 but this

Positive on the other side is that 3 of these 4 pages use already the same term grott beta branch (2.8) and always in the same way which I really like.
This is my example which leads to a lot of confusion and I have written that down for egguy as an issue.

I suggest that you and egguy and muppet3000 find a way to unify the naming sheme and version management and try to avoid "beta" as part of the title cause it lets look the solution a lot weaker in my opinion (which is not weak - rather strong cause at the beginning I had expected a lot more issues which I was used to as a growatt api user before).

And that you find a convention that the "GROTT 3.0 - HA Edition" aka Server uses the same versions you are using that everyone knows what it is based on cause from the grott 2.8 to 2.8.3 a lot of things have changed or improved like MIN support if I remember right.

And then they can use their own versioning so GROTT 3.0 - HA Edition V 0.1.7" for example . Then I would instantly know what version I am using. If I come to your grott page and open an issue I can tell your for sure what grott I am using which I can not right now at least not for sure. 2.8 something and maybe 2.8.3

I needs a cleaner and simple picture where everyone can choose what he wants or needs especially between the plug in solution by egguy and the Grott HA integration muppet 3000 is providing.

Would help a lot if these simple things can be changed or streamlined till version 3 to make things easier from then on.
THanks a lot. I like Grott as it is and the continous improvements even though they might feel slow from time to time but (looking from the past waiting for the arrival of the official mic and min support which is now there or at least the workarounds work) . If I find workarounds I document those clearly that others can see that grott is great even for me as newbie with over a dozen growatt inverters in total in 3 installations (all HA).

Hat off, great things you have made possible and of cause muppet3000 and egguy aka Chris and Etienne if I remember right.

[Sicos1977]

I hope this topic is the right place to post my question (because I need some help). I have the same issue with the growatt servers and decided to use your solution.

I think I have setup everything correctly but I don't get any data. The growatt dongle keeps saying there is no connection. Do you have any idea what I'm doing wrong?

[Sicos1977]

Well never mind ignore my question. I had to open the port on the firewall on my server. So for other people that ran into this problem and when you are on Windows.

Add the following firewall rule (assuming that you are using port 5279).

Choose what is needed

Give it a name

And you are done

When everything is working you will get messages like this when you start grott from the command line (for testing)

[Sicos1977]

I Home assistant you first need to add Grott. You can use HACS for that or just follow this --> https://github.com/muppet3000/homeassistant-grott

Don't forget to restart Home Assistant otherwise you wont see the new addon

When added you can add a new integration

Just leave this as it is.

At first grott will say that no devices and entities are found... just wait a minute and suddenly it will find your Growatt device (when everything is configured correctly)

When you click on the entities you will see something like this.

And gone is your problem with those horrible growatt servers (and horrible mobile app)

[Sicos1977]

You now can add this information to your energy dashboard.

Click on add below Solar panels (I think it is called this in an English Home Assistant)

I think you need Energy totday, I'll update this post when I'm sure about this

[triqster]

For me the today/vandaag part didn't work as it seems to reset around 07:00 AM CET resulting in faulty dashboard. Think it's best sticking to the total one as that doesn't do a daily reset ad odd hours.

[FreezyExp]

My docker server and/or proxy seems to get stuck every few days, restarting the docker is the only thing to get it moving again.

Aside from that, once setup it just does what it needs to.
Having more options to finetune the settings could be nice.

Perhaps adding a way to request firmware updates from the growatt servers could be added (i have never let the shinelan-x or inverter update, as they are not connected to the internet at all, but I am curious if there might be updates)

[FreezyExp]

One minor issue i also ran into was unclear port usage, but that can easily be solved by adding a comment into docker-compose like:

Inverter port
Grott status page (optional, will expose settings in the future)

And documenting this in the installation as well.

Inverter -> proxy port X -> grott -> grott server port Y or growatt
With a optional status/setting page.

[johanmeijer]

Your freeze of the docker container is strange but hopefully we can solve that.

Requesting for a firmware update should be possible with the api. But then we have to trace which requests are being sent/with registers are set. First it needs to point to the right firmware location and then start the update needs to be initiated.

Or indeed make it possible to temporary switch to growatt. But the will be something for Grott 3.0 where the server and proxy are more integrated.

[jmalm]

@FreezyExp , have you published your docker image or Dockerfile anywhere?

@johanmeijer , that's one thing that I would suggest could make it easier for people to start using grott - to have docker images publicly available for running grott.

[jmalm]

Sorry, I hadn't realized that there already is an official docker image. Don't know how I missed it when I set it up about a year ago. Will try it, as well as grottserver.

[triqster]

Something worth mentioning in the docs is that you'll have to restart the ShineLink after configuring the grott endpoint. As only after restarting the data will be send to the new configured endpoint.

Something I can assume that would help some less experienced users out is to provide a container that already includes the grott_ha extension including a docker-compose example and make sure the latest tag is actually updated on every release as that one seems to be broken.

At some point I might actually build a helm chart for deploying this on Kubernetes and contribute it here.

[PeterKawa]

Hello Johan!

First of all thanks a mil for sharing your project. While the Growatt servers are down/broken again, I found your repo at the HA forum.
I managed to install it by myself :P but 2 things took me some time to understand/find.

While it seems obvious I didn't get the difference between "Growatt server" and "server". Only today I realized with 'server' you mean the 'in between box' proxy/sniffer computing device. To me it would be clearer if you named it just "your server" "your Rpi" or the like.
When it finally struck me, and after added the local IP address of my server to the server IP field, just below the Growatt Server Domain field in the ShineWifi advanced settings:
The magic happened, MQTT topic was received, and I could add the device and sensors to HA.
Now I wanted to add grott as a service. It didn't work, and by the error message I found out there's 2 entries for the user name, 'pi', and one user folder by default, which I couldn't find any info for.
After editing the user name and user folder it started w/o issues.
Snippet of grott.service

SyslogIdentifier=grott
Type=simple
User=username_here

WorkingDirectory=/home/username_here/grott/
ExecStart=-/usr/bin/python3 -u /home/username_here/grott/grott.py -v

Also this means I can confirm it works fine with a Growatt 2000 ShineWifi,
while not recognized, it's added as default type:

	 - Possible Inverter serial TEC38xxxxx
	 - Inverter serial not recognised - using inverter type default

Oh I use a linux Debian 12 NUC by the way (no Docker this time)

Thanks again,
Best,
Peter

[jar349]

A few things:

• When you want to use something like influxdb, you also need pytz and influxdb-client packages, but the documentation does not say so.
• Configuration for using InfluxDB v1 and v2 are mixed together and not all values are required. Consider two separate config stanzas: [Influxdb] and [Influxdb2].
• Grott prints out the values that it got from the Inverter. Those values are reasonable. When you see the values sent to InfluxDB, they are not the same. I looked at the code and saw that - when it prints out values - grott modifies some values it gets by dividing things by an amount defined in the message type's configuration/schema. It doesn't do this for the values it sends to InfluxDB. Either way is ok, but the documentation should say why there's a difference.
• The project is missing common things that a professional Python developer would expect to see in a project:
  • usage of a package manager such as poetry/pipenv, or at least a requirements.txt file
  • more object-oriented code that would make the codebase more understandable (or at least the extraction of long chunks of code into their own functions)
  • the use of GitHub tags and releases to demarcate new releases.
  • a GitHub Action that kicks off every time there is a new tag and automatically builds a docker image and pushes it to docker hub.

[johanmeijer]

Thanks. In short:

1. Ok.
2. Will consider that in next release (but will give problems maybe with backwards compatibility)
3. Ok. Can make that more clear. I have chosen to sent the unformatted values to MQTT and Influxdb. That seem to be a more clean approach to me.
4. Hmm. It think I understand what you mean but it is a hobby project for me ( I am a professional IT-er, not a professional programmer).
   As a infra-architect one of the goals is to also get some practical experiences (and that works for me).
   I have chosen to publish it so others can use (enjoy) it (and is rather populair I think).
   But I will look at your tips and see if I can use them. Be aware I started with programming Algol, Fortran, Basic, Pascal and assembler in the 80's. Practising good object orient programming is a challenge for me. Hopefully it does not border you to much. To be honest I have seen a lot of beautiful written code oriented programs that did not do to job that well.

[jar349]

As a infra-architect one of the goals is to also get some practical experiences (and that works for me).
I have chosen to publish it so others can use (enjoy) it (and is rather populair I think).

I'm very pleased to find this project and have it working for me. I'm grateful to you for your efforts. My comments weren't meant to denigrate you or your work. The value of writing code that is closer to what is considered "usual" is that it makes your codebase easier to approach and understand since it will be more familiar. And that's the overall topic of this discussion: "Help making Grott more user friendly".

But I will look at your tips and see if I can use them. Be aware I started with programming Algol, Fortran, Basic, Pascal and assembler in the 80's.

Are you willing to entertain a PR or two that uses python's logging, or poetry/pipenv for package management? If so, I'll craft a few.

To be honest I have seen a lot of beautiful written code oriented programs that did not do to job that well.

It's always possible to write poor code. object-orientation isn't a magic bullet.

[johanmeijer]

Hi @jar349. Thank you for remarks. I really appreciate your improvement remarks and will take them in account.

Improve logging (with python logging) is one of the points of todo list.
Poetry and pipenv I have to look into but will certainly consider it.

As said one of my goals is to learn. And for me that also means that I want to do the things myself. I really appreciate that enthusiastic people want to improve Grott. But I am not ready for letting the project go and hand it fully over to the community.
I am afraid that not knowing the detail of the tool will hamper me with solving issues and helping people to use it.

Maybe in the future I will become a kind of Linus Thorvalds who is only controlling the changes of his project ;). For now I still want to do it myself.

I am not sure if I offend enthusiastic people with this statement. I really like the improvement advices and remarks, but for now I have to make the improvements myself. Maybe it is a little bit autistic, but that is not really strange in IT world I think.

[egguy]

Hi,

For addressing the first point, you could start doing this by adding a requirements.txt. This is a good, easy, and standard way to list dependencies.

This is a txt file listing the dependencies and the versions, e.g.:

paho-mqtt>=1.6.1,<1.7.0
requests==2.31.0
influxdb==5.3.1

The file is placed at the root of the project. This allows people to install the required dependencies using pip with a simple pip install -r requirements.txt

For the value sent to influxdb, you could also send the formatted metrics and the _raw (e.g. vpv1 and vpv1_raw) metrics, allowing people to post process them if they want and this also helps to understand this is the unprocessed value.

I would love some tagging of the releases, this would help greatly with the packaging. (along with the requirements.txt)

Great work, and thanks for grott.

[CK0y0TE]

Hi,
Just some info for improving the wiki for setting up the datalogger

• for me, accessing the ShineWifi-X its webpage in AP mode is impossible with firmware 3.1.0.5.
• "datalogYYYYMMDD" does work and "growatt/oss + YYYYMMDD" does not work on my Shine X wifi datalogger with firmware 3.1.0.5. (BTW that passwd/key , it is in the referred PDF at the growatt site)

[johanmeijer]

Thanks. Strange that you cannot access the Shinewifi in AP mode. How do you setup the Stick initially then?
I have a Shinewifi-S but have to reset the stick before I can access it (in AP mode). In normal operation mode the Stick is unreachable.

Strange that datalogYYYYMMDD works for yoou while this was the "old" key.

[CK0y0TE]

By using the android app.
It enters "app mode" after "reset button" and scanning QR. I can connect to its wifi with the passwd but the browser wont open anything on the ap (router) ip. I ran a simple android tool "fing" to scan for ports too, nothing. Used laptop (only one client allowed at a time) and same thing, tried different browsers .
Continuing with regular setup from the android app is all swell.
BTW, today the key is "growatt..." again. AND what I ment in the first post is the key to use in the server.growatt page to modify IP/port.

[johanmeijer]

what I normal do is reset the wifi stick. Connect with WIFIi (eg your phone/laptop) to the wifistick AP (network ESP.....). After connect to the WIFI netwerk of the stick you can go to the home page of the stick 192.168.4.1 via a browser. Then it should be possible to configure.

[ZonPV]

H Johan,

Glad with sharing your software! Thanks for your time and patience to help us so personal.
Happy to get an oppurtunity to open a discussion!
Good to share your knowledge in a Wiki instead of only a kind of read.me as often on this site.

To answer your question for tips to make Grott more userfriendly:

Tip 1. For me it was difficult to understand what was meant with different sections during reading the grott.ini. Especially related to the different versions/functions of the software. When do you fill in IP and Portnumber in the Generic section or in the Growatt-section? And what IP and port? Maybe a different version of grott.ini for Grott Proxy and Grott Server?

Tip 2. A list of errors and their explanation, cause and maybe solution to solve.
I understand that is difficult with all those different hard- and softwareplatforms and versions..

[theminch]

Hi Johanmeijer, would you be able to help me? Grott when installed via Add-Ons Store is not passing the MQTT username/password
In short, I have removed, uninstalled and reinstalled grott about 100 times and nothing I do will get Grott to read from and implement the grott.ini file which has this in it.

nomqtt = False
ip = localhost
port = 1883
retain = True
topic= homeassistant/growatt
auth = True
user = mqtt
password = my_password

Mosquitto logs show:
2024-07-22 17:11:22: New connection from 172.30.32.1:55076 on port 1883.
error: received null username or password for unpwd check

The Grott log doesn't give me much and just tells me about the environmental variable which I cannot change as they are fixed in the docker container.
[18:02:34] INFO: Preparing to start...
**********************************************************************************
- Grott - libscrc not installed, no CRC checking only record validation on length!
**********************************************************************************
Grott Growatt logging monitor : 2.8.3

Grott process configuration file

Grott process environmental variables

Pulled 'gmode=proxy' from the environment

Pulled 'gverbose=false' from the environment

Pulled 'ginvtype=default' from the environment

Pulled 'gsendbuf=true' from the environment

Grott IP address env invalid

Pulled 'gnomqtt=False' from the environment

Pulled 'gmqttip=172.30.33.0' from the environment

Pulled 'gmqttport=1883' from the environment

Pulled 'gmqtttopic=energy/growatt' from the environment

Pulled 'gmqttauth=True' from the environment

Pulled 'gmqttuser=addons' from the environment

Pulled 'gmqttpassword=ieph8iw0utahg8oozaiWoo2wet9othaich0aiRaedeeg3aevahs4thue9Aalieth' from the environment

Pulled 'gifpassword=OBFUSCS' from the environment

Pulled 'giforg=tOBFSUC' from the environment

Pulled 'gextension=True' from the environment

Pulled 'gextname=grottext.ha' from the environment

Pulled 'gextvar={"ha_mqtt_host": "core-mosquitto", "ha_mqtt_port": "1883", "ha_mqtt_user": "addons", "ha_mqtt_password": "ieph8iw0utahg8oozaiWoo2wet9othaich0aiRaedeeg3aevahs4thue9Aalieth", "ha_mqtt_retain": True}' from the environment

Grott override settings if set in commandline

Grott process json layout files

Grott proxy mode started
**********************************************************************************
- Grott - libscrc not installed, no CRC checking only record validation on length!
**********************************************************************************
Hostname : 1ef934e0-grott
IP : 172.30.33.6 , port : 5279

[johanmeijer]

I am not totally up to speed with the HA extension (I am not running HA yet and have not created this extension). Which docker container are you using? Is this the one provided in the HA library ?

Maybe @egguy can help us here.

[theminch]

If I change the topic in grott.ini then it pulls through to the grott log but no matter what I put for ip= the log shows 172.30.33.0

The addon configuration seems to make no difference.

I've installed it using the standard ha addon, I don't know much about docker other than there is no point in changing anything in the docker container.

[theminch]

I am not totally up to speed with the HA extension (I am not running HA yet and have not created this extension). Which docker container are you using? Is this the one provided in the HA library ?

Maybe @egguy can help us here.

Thanks Johan and Etienne. I have a screen recoding video I can share showing the problem but I don't want to share it publically due to passwords. I've requested a connection with Etienne on LinkedIn and can DM the video link there.

[PedroKTFC]

Are you running Grott Beta branch (2.8)? That's the add-on I use. I don't have an ini file, I simply use the add-on configuration and mine is as follows:

data_path: /config/grott
gmode: proxy
grott_mqtt: true
ginvtype: SPH
ginvtypemap: ""
retain: false
verbose: false
ha_plugin: true
mqtt: {}
ggrowattport: 5781
ggrowattip: 172.16.0.125

And you are running a Mosquitto broker I hope?

I also use the Grott integration which simplifies access to the entities.

[theminch]

...and with that addon config and having deleted grott.ini I get this in the logs. You'll see the IP and user don't match what's in the configuation and I have no idea where it is pulling this info from, other than the Docker image which clearly I cannot easily change.

_MQTT:
nomqtt False
mqttip: 172.30.33.0
mqttport: 1883
mqtttopic: energy/growatt
mqttmtopic: False
mqttmtopicname: energy/meter
mqttinverterintopic: False
mqtttretain: True
mqtttauth: True
mqttuser: addons
mqttpsw: secret

[egguy]

HA add-ons run in docker container, and have internal systems for discovery, network, and configuration .

For the user / password for mqttt, when the fields are not replaced by the a grott.ini or a custom configuration, the config is pulled from HA: https://developers.home-assistant.io/docs/add-ons/communication/#services-api .
This is why there is an auto-generated and unique password in this field.

It is the same idea for the internal IP (docker to docker) of the mosquito broker.

If you enable the plugin and keep the configuration as a default, it should auto-provide the info (grott MQTT and grott HA plugin)

If you want to do it manually, The HA plugin for grott has it own configuration in the .ini section if you wish to configure it manually: https://egguy.github.io/grott-ha-plugin/#usage

[theminch]

Hi @egguy.
I've

1. removed the container docker rm 5e1bfc40db97
2. removed the image docker rmi 39e8258665c8
3. Rebooted Home Assistant (not restarted)
4. Installed Grott - Grott stable branch (2.8) Current version: 0.1.10 (Changelog) with this link https://github.com/egguy/addon-grott/tree/v0.1.10 - ... and set the addon to Start On Boot (via toggle switch on Info tab)
5. Rebooted Home Assistant (not restarted)

The addon does not create anything in /config/ it creates this file structure /addon_configs/1ef934e0_grott/grott/ with a 0 Byte (empty) grott.ini
➜ / ls /config/grott
ls: /config/grott: No such file or directory
➜ / ls -l /addon_configs/1ef934e0_grott/grott/
total 0
-rw-r--r-- 1 root root 0 Jul 24 20:40 grott.ini

Question 1: As I am using the Mosquitto Broker addon and/or the grott_ha extension, should this work straight out of the box with no further config required?
Question 2: Should I update the data_path in the addon configuration to reflect "/addon_configs/1ef934e0_grott/grott/" ...? -- Notably I've tried putting a grott.ini file here and also in /config/grott and various other places but the addon seems to ignore it.
Question 3: Where is the addon getting Pulled 'gnomqtt=True' from the environment from and should I change this to False, and if so how?
Questions 4: The addon configuation has this yaml below as default after fresh install, notable grott_mqtt is false, is this correct? will this publish to my Mosquitto Broker AddOn in HA? the MQTT log is at the bottom of this post and I'm pretty sure it's not seeing grott sockets attempting to come in as I assume from the bottom of the grott addon log that the grott source socket is 172.30.33.6:5279 and I don't see this in the MQTT log.
Question 5: what would you recommend doing now to publish to my MQTT broker specifying IP, Port, Auth, Certs, Username and Password..?

Thanks for your assistance, happy to work through this but need some direction.

GROTT ADDON CONFIGURATION:
data_path: /config/grott
gmode: proxy
grott_mqtt: false
ginvtype: default
ginvtypemap: ""
retain: false
verbose: false
ha_plugin: true
mqtt: {}

GROTT ADDON LOG:
s6-rc: info: service s6rc-oneshot-runner successfully stopped
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service base-addon-banner: starting

---

Add-on: Grott stable branch (2.8)
The Growatt inverter monitor with current HA plugin integrated

Add-on version: 0.1.10
You are running the latest version of this add-on.
System: Home Assistant OS 12.4 (aarch64 / raspberrypi4-64)
Home Assistant Core: 2024.7.3
Home Assistant Supervisor: 2024.06.2

Please, share the above information when looking for help
or support in, e.g., GitHub, forums or the Discord chat.

s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
[21:43:21] INFO: Preparing to start...
**********************************************************************************
- Grott - libscrc not installed, no CRC checking only record validation on length!
**********************************************************************************
Grott Growatt logging monitor : 2.8.3

Grott process configuration file

Grott process environmental variables

Pulled 'ginvtype=default' from the environment

Pulled 'gnomqtt=True' from the environment

Pulled 'gextension=True' from the environment

Pulled 'gextname=grottext.ha' from the environment

Pulled 'gextvar={"ha_mqtt_host": "core-mosquitto", "ha_mqtt_port": "1883", "ha_mqtt_user": "addons", "ha_mqtt_password": "ieph8iw0utahg8oozaiWoo2wet9othaich0aiRaedeeg3aevahs4thue9Aalieth", "ha_mqtt_retain": False}' from the environment

Grott override settings if set in commandline

Grott process json layout files

Grott proxy mode started
**********************************************************************************
- Grott - libscrc not installed, no CRC checking only record validation on length!
**********************************************************************************
Hostname : 1ef934e0-grott
IP : 172.30.33.6 , port : 5279

MQTT LOGS:
2024-07-24 21:41:37: New connection from 172.30.32.2:34686 on port 1883.
2024-07-24 21:41:37: Client closed its connection.
2024-07-24 21:42:08: New connection from 172.30.32.1:57492 on port 1883.
error: received null username or password for unpwd check
2024-07-24 21:42:08: Client auto-65E81AC0-464B-DC39-6E5E-FBF315A4FB4F disconnected, not authorised.
2024-07-24 21:42:27: Connecting bridge bridge-01 (eu1.cloud.thethings.network:8883)
2024-07-24 21:42:31: New connection from 192.168.0.55:55034 on port 1883.
2024-07-24 21:42:31: New client connected from 192.168.0.55:55034 as sh.htg3.bbfc.55.hum_and_temp (p2, c1, k60, u'mqtt').
2024-07-24 21:42:41: Client sh.htg3.bbfc.55.hum_and_temp closed its connection.
2024-07-24 21:42:44: New connection from 192.168.0.55:55037 on port 1883.
2024-07-24 21:42:44: New client connected from 192.168.0.55:55037 as sh.htg3.bbfc.55.hum_and_temp (p2, c1, k60, u'mqtt').
2024-07-24 21:43:37: New connection from 172.30.32.2:39794 on port 1883.
2024-07-24 21:43:37: Client closed its connection.
2024-07-24 21:43:41: New connection from 192.168.0.80:5411 on port 1883.
2024-07-24 21:43:41: New client connected from 192.168.0.80:5411 as shellybutton1-34945476917C (p2, c1, k60, u'mqtt').
2024-07-24 21:43:58: Connecting bridge bridge-01 (eu1.cloud.thethings.network:8883)

[theminch]

@egguy I am pretty sure this is the Grott docker trying to connect to the Mosquitto broker and failing because no user/pass are being received. What would you suggest next to debug, isolate the issue and what potential fixes could I try..?

##> Mosquitto MQTT Broker Log
2024-07-25 09:46:53: New connection from 172.30.32.1:41706 on port 1883.
error: received null username or password for unpwd check
2024-07-25 09:46:53: Client auto-6FE940FB-5EDA-9E7C-2EC5-B6EBE1354CCD disconnected, not authorised.

##> Here is the Grott add-on YAML
data_path: /config/grott
gmode: proxy
grott_mqtt: true
ginvtype: default
ginvtypemap: "{"NWCTCKJ07E": "sph"}"
retain: true
verbose: true
ha_plugin: false
mqtt: {}
gmqttauth: true
gmqttpassword: grottpw
gmqttuser: grott

##> Note: grott.ini file is present but empty.
➜ / find / -name "grot"
/etc/logrotate.d
/addon_configs/1ef934e0_grott
/addon_configs/1ef934e0_grott/grott
/addon_configs/1ef934e0_grott/grott/grott.ini
/homeassistant/custom_components/grott
➜ / cat /addon_configs/1ef934e0_grott/grott/grott.ini
➜ / head /addon_configs/1ef934e0_grott/grott/grott.ini
➜ / # Nothing in grott.ini
➜ /

##> Here is the Grott AddOn Log
'bms_packinfo': {'value': 1066, 'length': 2, 'type': 'num', 'divide': 1}, 'bms_usingcap': {'value': 1070, 'length': 2, 'type': 'num', 'divide': 1}, 'bms_fw': {'value': 1074, 'length': 2, 'type': 'num', 'divide': 1}, 'bms_mcuversion': {'value': 1078, 'length': 2, 'type': 'num', 'divide': 1}, 'bms_commtype': {'value': 1082, 'length': 2, 'type': 'num', 'divide': 1}}

Grott external record whitelist 'recwl.txt' not found

Grott records whitelisted : {'0120', '5104', '5019', '5103', '5016', '5004', '501b', '0150', '5050', '0105', '0119', '5129', '5150', '5229', '5003', '5005', '5119', '0103', '5216', '5219', '0104', '0116', '5250', '5116', '5105'}

Grott settings:

_Generic:
version: 2.8.3
verbose: True
trace: False
config file: /config/grott/grott.ini
minrecl: 100
decrypt: True
compat: False
invtype: default
invtypemap: {'NWCTCKJ07E': 'sph'}
include_all: False
blockcmd: False
noipf: False
time: auto
sendbuf: True
timezone: local
valueoffset: 6
offset: 6
inverterid: automatic
mode: proxy
grottip default
grottport 5279
_MQTT:
nomqtt False
mqttip: 172.30.33.0
mqttport: 1883
mqtttopic: energy/growatt
mqttmtopic: False
mqttmtopicname: energy/meter
mqttinverterintopic: False
mqtttretain: False
mqtttauth: True
mqttuser: addons
mqttpsw: secret
_Growatt server:
growattip: 47.91.67.66
growattport: 5279
_PVOutput:
pvoutput: False
pvdisv1: False
pvtemp: False
pvurl: https://pvoutput.org/service/r2/addstatus.jsp
pvapikey: yourapikey
pvinverters: 1
pvsystemid: systemid1
_Influxdb:
influx: False
influx2: False
database: grottdb
ip: localhost
port: 8086
user: grott
password: secret
organization: grottorg
bucket: grottdb
token: secret
_Extension:
extension: False
extname: grottext
extvar: {'none': 'none'}

Grott proxy mode started
**********************************************************************************
- Grott - libscrc not installed, no CRC checking only record validation on length!
**********************************************************************************
Hostname : 1ef934e0-grott
IP : 172.30.33.6 , port : 5279

[theminch]

@egguy @johanmeijer I am able to publish ok myself using Mosquitto and Paho (see below) so it does look like grott is just not passing the credentials to the broker as I am seeing this in the Mosquitto logs, any ideas on how to troubleshoot.

2024-07-25 15:57:10: New connection from 172.30.32.1:41650 on port 1883.
error: received null username or password for unpwd check
2024-07-25 15:57:10: Client auto-263613CD-741D-BE82-E8E8-8E111B47F6C4 disconnected, not authorised.

This publishes successfully using Mosquitto/Terminal--->

mosquitto_pub -h 192.168.0.5 -p 8883 -u "grott" -P "grottpw" --cafile /path/to/ca.crt --cert /path/to/client.crt --key

This publishes successfully using Paho/Python--->
import paho.mqtt.client as mqtt
import ssl
mqtt_broker = '192.168.0.5'
mqtt_port = 1883
topic = 'energy/growatt'
message = 'your_message_here'
username = 'grott'
password = 'grottpw'
client = mqtt.Client()
client.username_pw_set(username, password)
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected successfully")
else:
print(f"Connect failed with code {rc}")
client.on_connect = on_connect
client.connect(mqtt_broker, mqtt_port, 60)
client.publish(topic, message)
client.disconnect()
print(f"Message '{message}' published to topic '{topic}'")

[Sicos1977]

While grott is a great project I switched to using modbus on my growatt inverter. This way you just can get all the values from you Growatt inverter without needing something in between like Grott

[Sicos1977]

You can use this HA integration when you get all your values through the modbus connector on your inverter.

https://github.com/wills106/homeassistant-solax-modbus

The only "downside" is that you have to connect a modbus cable to your inverter and add a modbus to ethernet converter.

[theminch]

HI Sicos1977. Connect a cable, what, with actual wires ??!! No thanks, there must be a solution to have grott use the configuration that I want it to use rather than pulling a random config and vairables from somewhere.
I appreciate your suggestion though, I did look at this two years ago and decided I didn't want hardwires.

I should point out that grott was working perfectly fine and then randomly stopped working a few weeks ago.
I tried to debug/fix it, had no luck, then uninstalled it, reinstalled it, configured grott.ini as before and the addon but no dice.

[typxxi]

yes and no.
I like the idea and I am starting to use the shinebus software to get familiar with modbus and the adapter.
I am using this with the RS-485 port cause the first one is used by the METER and the other one is can bus connection to the JK BMS battery .
For now I was able to read and use the Shinebus Windows software and I need a solution cause growatt SPH 4600 has a fixed modbus adress 001 and our eastron meter had to become 002 while now I am challenged to get those or both running with the pulsar plus wallbox from the manufacturer wallbox which is running modbus rtu.

I guess I will have to reuse "split the modbus cable" between the smartmeter and SPH inverter which would end up in this

But with all my other 7 growatt inverter I will not wanna got that way at least for now. Who wants to deal with 8 cables and USB adapters?
Maybe there is an arduino or esp solution for that as a kind of collector but right now I will stay with grott cause it supports the cloud and also the shinephone app some older family members have learned to love. Above 80 years you are not so keen to learn everything over and over again and I am quite happy that htey made the jump and replaced the diesel with an EV 5 years ago and also the diesel central heating system with a heatpump and 2 split ac as heater for the bigger living rooms.

THerefore yes , but only for a limited amount and in addition to the grott server we use mainly.

[triqster]

First let me start with saying thank you Johan for creating Grott, I've made good use out if it for a while!

I've recently switched from using grott for a long time to hardwire modbus, it's way more reliable, faster and you get more data. The data can be polled nearly realtime from the inverter (3 seconds in my case). And you get information like temperature and output per string, which can be used to detect one of the solar pannels failing i.e.

all you really need is the correct modbus connector, a esp and a rs485.

[CK0y0TE]

I did mess with it a lot too 1 setup a user passwd in web interface 2 setup in webinterface to match mosquito. Op wo 24 jul. 2024 22:51 schreef theminch ***@***.***>:

…

Hi @egguy <https://github.com/egguy>. I've 1. removed the container docker rm 5e1bfc40db97 2. removed the image docker rmi 39e8258665c8 3. Rebooted Home Assistant (not restarted) 4. Installed Grott - Grott stable branch (2.8) Current version: 0.1.10 (Changelog) with this link https://github.com/egguy/addon-grott/tree/v0.1.10 - ... and set the addon to Start On Boot (via toggle switch on Info tab) 5. Rebooted Home Assistant (not restarted) The addon does not create anything in /config/ it creates this file structure /addon_configs/1ef934e0_grott/grott/ with a 0 Byte (empty) grott.ini ➜ / ls /config/grott ls: /config/grott: No such file or directory ➜ / ls -l /addon_configs/1ef934e0_grott/grott/ total 0 -rw-r--r-- 1 root root 0 Jul 24 20:40 grott.ini Question 1: As I am using the Mosquitto Broker addon and/or the grott_ha extension, should this work straight out of the box with no further config required? Question 2: Should I update the data_path in the addon configuration to reflect "/addon_configs/1ef934e0_grott/grott/" ...? -- Notably I've tried putting a grott.ini file here and also in /config/grott and various other places but the addon seems to ignore it. Question 3: Where is the addon getting Pulled 'gnomqtt=True' from the environment from and should I change this to False, and if so how? Questions 4: The addon configuation has this yaml below as default after fresh install, notable grott_mqtt is false, is this correct? will this publish to my Mosquitto Broker AddOn in HA? the MQTT log is at the bottom of this post and I'm pretty sure it's not seeing grott sockets attempting to come in as I assume from the bottom of the grott addon log that the grott source socket is 172.30.33.6:5279 and I don't see this in the MQTT log. Question 5: what would you recommend doing now to publish to my MQTT broker specifying IP, Port, Auth, Certs, Username and Password..? Thanks for your assistance, happy to work through this but need some direction. GROTT ADDON CONFIGURATION: data_path: /config/grott gmode: proxy grott_mqtt: false ginvtype: default ginvtypemap: "" retain: false verbose: false ha_plugin: true mqtt: {} GROTT ADDON LOG: s6-rc: info: service s6rc-oneshot-runner successfully stopped s6-rc: info: service s6rc-oneshot-runner: starting s6-rc: info: service s6rc-oneshot-runner successfully started s6-rc: info: service base-addon-banner: starting ------------------------------ Add-on: Grott stable branch (2.8) The Growatt inverter monitor with current HA plugin integrated Add-on version: 0.1.10 You are running the latest version of this add-on. System: Home Assistant OS 12.4 (aarch64 / raspberrypi4-64) Home Assistant Core: 2024.7.3 Home Assistant Supervisor: 2024.06.2 Please, share the above information when looking for help or support in, e.g., GitHub, forums or the Discord chat. s6-rc: info: service base-addon-banner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service base-addon-log-level: starting s6-rc: info: service fix-attrs successfully started s6-rc: info: service base-addon-log-level successfully started s6-rc: info: service legacy-cont-init: starting s6-rc: info: service legacy-cont-init successfully started s6-rc: info: service legacy-services: starting s6-rc: info: service legacy-services successfully started [21:43:21] INFO: Preparing to start... ********************************************************************************** - Grott - libscrc not installed, no CRC checking only record validation on length! ********************************************************************************** Grott Growatt logging monitor : 2.8.3 Grott process configuration file Grott process environmental variables Pulled 'ginvtype=default' from the environment Pulled 'gnomqtt=True' from the environment Pulled 'gextension=True' from the environment Pulled 'gextname=grottext.ha' from the environment Pulled 'gextvar={"ha_mqtt_host": "core-mosquitto", "ha_mqtt_port": "1883", "ha_mqtt_user": "addons", "ha_mqtt_password": "ieph8iw0utahg8oozaiWoo2wet9othaich0aiRaedeeg3aevahs4thue9Aalieth", "ha_mqtt_retain": False}' from the environment Grott override settings if set in commandline Grott process json layout files Grott proxy mode started ********************************************************************************** - Grott - libscrc not installed, no CRC checking only record validation on length! ********************************************************************************** Hostname : 1ef934e0-grott IP : 172.30.33.6 , port : 5279 MQTT LOGS: 2024-07-24 21:41:37: New connection from 172.30.32.2:34686 on port 1883. 2024-07-24 21:41:37: Client closed its connection. 2024-07-24 21:42:08: New connection from 172.30.32.1:57492 on port 1883. error: received null username or password for unpwd check 2024-07-24 21:42:08: Client auto-65E81AC0-464B-DC39-6E5E-FBF315A4FB4F disconnected, not authorised. 2024-07-24 21:42:27: Connecting bridge bridge-01 (eu1.cloud.thethings.network:8883) 2024-07-24 21:42:31: New connection from 192.168.0.55:55034 on port 1883. 2024-07-24 21:42:31: New client connected from 192.168.0.55:55034 as sh.htg3.bbfc.55.hum_and_temp (p2, c1, k60, u'mqtt'). 2024-07-24 21:42:41: Client sh.htg3.bbfc.55.hum_and_temp closed its connection. 2024-07-24 21:42:44: New connection from 192.168.0.55:55037 on port 1883. 2024-07-24 21:42:44: New client connected from 192.168.0.55:55037 as sh.htg3.bbfc.55.hum_and_temp (p2, c1, k60, u'mqtt'). 2024-07-24 21:43:37: New connection from 172.30.32.2:39794 on port 1883. 2024-07-24 21:43:37: Client closed its connection. 2024-07-24 21:43:41: New connection from 192.168.0.80:5411 on port 1883. 2024-07-24 21:43:41: New client connected from 192.168.0.80:5411 as shellybutton1-34945476917C (p2, c1, k60, u'mqtt'). 2024-07-24 21:43:58: Connecting bridge bridge-01 (eu1.cloud.thethings.network:8883) — Reply to this email directly, view it on GitHub <#423 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEV42JLIVHWAA2FUUZXZU4DZOAHWVAVCNFSM6AAAAABLIXNW76VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMJUGIZTENQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[theminch]

Thanks. I've just used the addon web UI to add the config, here's the YAML it produced
data_path: /config/grott
gmode: proxy
grott_mqtt: true
ginvtype: default
ginvtypemap: "{"NWCTCKJ07E": "sph"}"
retain: true
verbose: true
ha_plugin: true
mqtt: {}
gextension: false
gmqttauth: true
gmqttinverterintopic: true
gmqttip: 192.168.0.5
gmqttpassword: grottpw
gmqttport: "1883"
gmqttuser: grott

And here's the AddOn log, you'll see none of the addon ui/yaml config has been applied and nothing is showing in MQTT or in HA as entities

'bms_fw': {'value': 1074, 'length': 2, 'type': 'num', 'divide': 1}, 'bms_mcuversion': {'value': 1078, 'length': 2, 'type': 'num', 'divide': 1}, 'bms_commtype': {'value': 1082, 'length': 2, 'type': 'num', 'divide': 1}}

Grott external record whitelist 'recwl.txt' not found

Grott records whitelisted : {'5104', '5016', '5250', '5050', '5116', '5150', '0119', '0104', '5216', '5004', '5129', '5019', '0120', '0150', '5103', '0105', '5003', '5219', '5229', '0103', '0116', '5105', '5005', '501b', '5119'}

Grott settings:

_Generic:
version: 2.8.3
verbose: True
trace: False
config file: /config/grott/grott.ini
minrecl: 100
decrypt: True
compat: False
invtype: default
invtypemap: {'NWCTCKJ07E': 'sph'}
include_all: False
blockcmd: False
noipf: False
time: auto
sendbuf: True
timezone: local
valueoffset: 6
offset: 6
inverterid: automatic
mode: proxy
grottip default
grottport 5279
_MQTT:
nomqtt False
mqttip: 172.30.33.0
mqttport: 1883
mqtttopic: energy/growatt
mqttmtopic: False
mqttmtopicname: energy/meter
mqttinverterintopic: False
mqtttretain: False
mqtttauth: True
mqttuser: addons
mqttpsw: secret
_Growatt server:
growattip: 47.91.67.66
growattport: 5279
_PVOutput:
pvoutput: False
pvdisv1: False
pvtemp: False
pvurl: https://pvoutput.org/service/r2/addstatus.jsp
pvapikey: yourapikey
pvinverters: 1
pvsystemid: systemid1
_Influxdb:
influx: False
influx2: False
database: grottdb
ip: localhost
port: 8086
user: grott
password: secret
organization: grottorg
bucket: grottdb
token: secret
_Extension:
extension: True
extname: grottext.ha
extvar: {'ha_mqtt_host': 'core-mosquitto', 'ha_mqtt_port': '1883', 'ha_mqtt_user': 'addons', 'ha_mqtt_password': 'ieph8iw0utahg8oozaiWoo2wet9othaich0aiRaedeeg3aevahs4thue9Aalieth', 'ha_mqtt_retain': True}

Grott proxy mode started
**********************************************************************************
- Grott - libscrc not installed, no CRC checking only record validation on length!
**********************************************************************************
Hostname : 1ef934e0-grott
IP : 172.30.33.6 , port : 5279

[egguy]

Looking at the generated configuration, this looks ok if you want to use the mosquitto addon. Do you have a custom broker?

Normally, after this, if your stick is configurer to send to HA you should start to see things like this:

 - Growatt packet received:
		  <socket.socket fd=5, family=2, type=1, proto=0, laddr=('172.30.33.7', 5279), raddr=('192.168.1.180', 56846)>

	 - Growatt packet received:
		  <socket.socket fd=4, family=2, type=1, proto=0, laddr=('172.30.33.7', 42804), raddr=('8.209.71.240', 5279)>
	 - Growatt data decrypted V2

If not, your stick is not sending the data.

[typxxi]

---

• Grott - libscrc not installed, no CRC checking only record validation on length!

---

Hostname : 1ef934e0-grott
IP : 172.30.33.6 , port : 5279

First things first:
I have read somewhere the inverter model needs to be uppercase and should have caused issues.

So change this
invtypemap: {'NWCTCKJ07E': 'sph'}
to
invtypemap: {'NWCTCKJ07E': 'SPH}
the documentation is also misleading cause it uses both ways.

my growattip is not 47.91.67.66 but
growattip: 8.209.71.240

And regarding your log: did you press the actualise button once you have started grott ?
It does not update automatically.

For me it looks OK but I am using the grott integration provided by muppet 3000 which then auto recognises the inverters and all the sensors which means it will create the missing ones and actualise every time new data arrive via mqtt.

AND: DO NOT FORGET TO MAKE THE CHANGE on THE STICK + FINALLY TO REBOOT HA AFTERWARDS.
In my experience the real HA reboot is necessary to get all the changes published properly. It has caused me a lot of headache back earlier this year when I had to start with 8 growatt inverters which finally worked out like a charm once the first was running properly.

Good luck but it is not that hard to get the grott server running at least as Grott HA addon from egguy which works great with the muppet 3000 HA integration for the sensors.