-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
After deployment of ExApp "Test Deploy" (nc_app_test-deploy) returns/shows: "Heartbeat check failed" and "Healtchecking" #300
Comments
Please provide the following in addition to the above:
|
|
|
The resulting Log File is about 60 MB (59944187 Jun 7 14:29 nc_appapi.log) and as I said it contains sensitive information. I am going to replace such sensitive information with dummy/fake information and then upload the log file |
Please find attache the log file |
What do you exactly need from Developer Console? |
The screenshot, hoping it contains the information you need |
It looks like the attachments are missing for the log file and the screenshot.
Sorry for the confusion. I'm looking for errors in the "Console" tab or the "Network" tab. |
Maybe github doesn't accept .png and .gz files? |
Seems to work for me. You can give it one more try by dragging the file in the text box or a link to the uploaded file elsewhere. |
I uploaded the files again... both in a zip file |
part 1/3 |
part 2/3 |
part 1/3 |
@architectonio Nice that you managed to get context chat running. Did you check if this was solved as well? For the attachments, I only see text messages this side (part 1/3, ...). You can use pastebin and imgur to upload the logs and screenshots and then paste a link here if the issue still persists. |
@kyteinsky |
It seems to work, however when I make a question in the NextCloud Context Chat, |
The same happens by trying to generate image like "Draw a red Rose in a brown pot" with "NC Assistant Generate Image". |
Sure, send it over at kyteinsky@gmail.com, I'll attach the files here. The issues might indicate that the app_api setup is not correct. Did it work before with image generation? |
Check your mailbox! |
@kyteinsky Any finding in the log file? |
sorry again for the late reply. The only relevant line in the log was:
... Error during request to ExApp context_chat_backend: cURL error 28: Failed to connect to context_chat_backend port 23001 after 134424 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://context_chat_backend:23001/loadSources ...
The "134423 ms" part is interesting. Can you check the php timeout in your php.ini config and increase it if it is too low? A good value could be 1800 to 3000 seconds. Also, I'd like to see the Test deploy modal. Please follow these steps to run a test deployment using AppAPI:
|
And here the ExApp logs: Started |
Could be a network issue. Can you click on the deploy daemon (not on the 3 dots) and then on "Verify connection" ? |
"Daemon connection successful" |
This is what gives back "docker ps" |
By the way, the reason why I couldn't attach any file in the comment seems to be related to Firefox. |
OK, I'll do as you suggested, but on Sunday in the late afternoon. Now I have to travel a little.... |
This is the Public IP Address I got (for today) by my ISP, on which points my domain |
I understand that, but that DNS names should be resolved to the local addresses(docker should do that) and not your public address. |
I created the network "nextcloud-aio" docker network inspect nextcloud-aio
I removed the daemon and redeployed as "nextcloud-appapi-dsp:2375". "docker ps" shows it up and running
A "nmap nextcloud-appapi-dsp -p2375" gives back:
And a ping shows my (currently) public IP Address
And NextCloud ExtApp Dashboard shows: "All ExApps are up-to-date. Default Deploy daemon is not accessible " I do not know is something is wrong on my Server Network Configuration, however everything else just runs smoothly, without any noticeable issue. |
What I also noticed is the fact that both "Test Deploy" and "Context Chat Backend" containers have no port exposed while all other containers are exposing a port. docker ps Another point I do not really catch is the "nextcloud-aio" network. |
Any news on this? |
Am I reading this correctly that the docker socket proxy cannot be on a remote host? Test deploy seems to fail for me, much as the original post here, because it cannot resolve the name http//:test-deploy:23000 |
I don't know if DSP must run on the local host, however my DSP runs on the local host and I tried with "host", "bridge" and also "nextcloud-aio". The issue remains the same, "Test Deploy" and "Context Chat Backend" are deployed but not reachable. |
It was mentioned as the assumption that you are using Nextcloud AIO - which has this custom network created for the AIO containers.
The purpose of the DSP - is to provide a secure access for AppAPI to docker via network, it can be local or remote.
Is there any logs or error that can give us a hint? Is there any errors related in system logs from docker ( For now I can't say more that was said before on how to investigate networking issues, since the daemon connection is fine and the deployment working, the issue is only in communication part between ExApp and NC, which is likely some specifics of certain system setup. I'll back to you as soon as find something. |
Well, on my end, for example, health check seems to be looking for
http://test-deploy:23000 and http://context-chat-backend:23000 which it
will never find because those are not in the same network as Nextcloud
(they are on the remote host at the IP address I gave Nextcloud when I set
up the socket proxy). So I’m not sure what can be done to have the health
check seek the right thing at the right location. It might help to be able
to better direct where the health check looks.
…On Tue, Jul 2, 2024 at 8:40 AM Andrey Borysenko ***@***.***> wrote:
@architectonio <https://github.com/architectonio>
Another point I do not really catch is the "nextcloud-aio" network.
It was mentioned as the assumption that you are using Nextcloud AIO -
which has this custom network created for the AIO containers.
I don't know if DSP must run on the local host
The purpose of the DSP - is to provide a secure access for AppAPI to
docker via network, it can be local or remote.
I also noticed that "Context Chat Backend" restarts every few seconds (by
observing the results of "watch -n 0.5 docker ps" ).
Is there any logs or error that can give us a hint? Is there any errors
related in system logs from docker (journalctl -u docker) or from Context
Chat Backend container?
For now I can't say more that was said before on how to investigate
networking issues, since the daemon connection is fine and the deployment
working, the issue is only in communication part between ExApp and NC,
which is likely some specifics of certain system setup. I'll back to you as
soon as find something.
—
Reply to this email directly, view it on GitHub
<#300 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APK5CFU6OLFFQNMCRGXGRITZKKUV7AVCNFSM6AAAAABI6PT7SOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBTGE4TQOBXGM>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
@andrey18106 Is there any logs or error that can give us a hint? Is there any errors related in system logs from docker (journalctl -u docker) or from Context Chat Backend container? As I wrote before, docker works perfectly, with no network or other issue. A "docker ps" gives back: Notice that the Test Deploy has no network or port Here the "docker network list" |
This is the running DSP (I removed Test-Deploy and Context_Chat Backend) which is reachableExApps installed: 0 |
I think they suggested not to use bridge because bridge won't look things up by container name. I set mine to master_default, but no difference in the behavior. |
I tried a lot of possible combinations, all without success.....I guess I invested at least 40 hours in testing. |
Same issue here. I have NC running in a VM (direct install, no docker). And I have docker for running this ExApp stuff. The daemon connection test is successful but the deployment test fails after a long time during the heartbeat check. No idea what else to try. |
I had exactly the same issue. |
Please create a separate issue with describing of your configuration.
Have you tried with the latest version 3.1.0 (where we fixed a critical bug with APCu), heartbeat still didn't work? If you tried and it still didn't work, as an option I can offer if you have the opportunity to give VPN access to the test environment where you can't do it, and we'll try to figure out what the reason might be. But with version 3.1.0 everything has already worked for most people, I hope that we can help you too. |
Yes I have tried with the latest version 3.1.0, however I just used the already deployed Docker Socket Proxy and I do not know if it affects in any way the AppAPI. Would be worth deleting the NextCloud Assistant, NextCloud Assistant Context Chat nad AppAPI and then reinstall again? |
Yes, I'm on 3.1.0 (I also updated NC to 29.0.8). One thing I noticed is that the nextcloud-appapi-dsp container becomes unhealthy relatively quickly. Not sure, if this has anything to do with the issue? (I downloaded the most recent image and updated the container but it still becomes unhealthy a minute after starting or so) |
You have a docker-socket-proxy address where it is listens. Look in the DB which port is assigned to the test-deploy application in the Try to do If you use This is literally what AppAPI does on heartbeat. To not this issue longer(it is already 65+ messages) - please create a separate issue with posted configs. |
This is what i get with https: curl https://127.0.0.1:2375/ -u app_api_haproxyuser:mytestpassword And this with http: curl http://127.0.0.1:2375/ -u app_api_haproxyuser:mytestpassword 401 UnauthorizedYou need a valid user and password to access this content. ** The DSP was deployed in this way: A docker ps shows: |
@architectonio Please correct the name of the user to |
curl http://127.0.0.1:2375/ -u app_api_haproxy_user:mytestpassword 403 ForbiddenRequest forbidden by administrative rules. |
There is no route in your request, it's not allowed, so the response is correct, and auth is passed. |
I assume this means that the AppAPI and everything that is deployed should work... |
Unfortunately the issue persists. A docker ps shows: And Nextcloud "You Apps" Dashboard shows both Apps with a Healtchecking loop. |
hi, I was testing with a proxmox setup where your issue was reproducible. Docker socket proxy was reachable, deployment succeeded but the heartbeat thing was stuck. It happens because "http" deployments as in your case @architectonio (if you're around) bind to the localhost (127.0.0.1) address (for security reasons) when the network setup would want them bound to another address.
The haproxy is bound to Thanks to @bigcat88 for the help! |
Describe the bug
After having deployed the ExApp "Test Deploy", the NextCloud External App Admin Interface shows a "Healthchecking"infinite loop as well as "Heartbeat check failed"
Steps/Code to Reproduce
Deploy the "Test Deploy" on NextCloud
Expected Results
Deployed without any issue
Actual Results
NextCloud External App Admin Interface shows a "Healthchecking"infinite loop as well as "Heartbeat check failed"
Setup configuration
Software
Hardware
result of: docker logs nc_app_test-deploy
Started
INFO: Started server process [1]
INFO: Waiting for application startup.
TRACE: ASGI [1] Started scope={'type': 'lifespan', 'asgi': {'version': '3.0', 'spec_version': '2.0'}, 'state': {}}
TRACE: ASGI [1] Receive {'type': 'lifespan.startup'}
TRACE: ASGI [1] Send {'type': 'lifespan.startup.complete'}
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:23000 (Press CTRL+C to quit)
INFO: Shutting down
INFO: Waiting for application shutdown.
TRACE: ASGI [1] Receive {'type': 'lifespan.shutdown'}
TRACE: ASGI [1] Send {'type': 'lifespan.shutdown.complete'}
TRACE: ASGI [1] Completed
INFO: Application shutdown complete.
INFO: Finished server process [1]
Started
INFO: Started server process [1]
INFO: Waiting for application startup.
TRACE: ASGI [1] Started scope={'type': 'lifespan', 'asgi': {'version': '3.0', 'spec_version': '2.0'}, 'state': {}}
TRACE: ASGI [1] Receive {'type': 'lifespan.startup'}
TRACE: ASGI [1] Send {'type': 'lifespan.startup.complete'}
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:23000 (Press CTRL+C to quit)
result of: docker volume inspect nc_app_test-deploy_data
[
{
"CreatedAt": "2024-05-28T10:36:06+02:00",
"Driver": "local",
"Labels": null,
"Mountpoint": "/var/lib/docker/volumes/nc_app_test-deploy_data/_data",
"Name": "nc_app_test-deploy_data",
"Options": null,
"Scope": "local"
}
]
The text was updated successfully, but these errors were encountered: