-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PM2 save doesn't seem to be working #1035
Comments
Can you try to do |
While pm2 is working and I run pm2 resurrect it spawns another process Thank you for your help |
Try this:
If it's not in the list we got an issue. Rebooting seems not to be the issue here. |
Doing the above steps results in having the app.js handled by pm2 website On Wed Feb 25 2015 at 19:01:55 Antoine Bluchet notifications@github.com
|
Any other ideas on why PM2 is having troubles resurrecting the processes after reboot? Thanks |
Same issue on debian 7. When I manually use |
same issue on beaglebone black debian. |
This happens to me, but only when the server runs out of memory. I get an out of memory error in my pm2 logs like so (which I know is my fault, not pm2's):
Then on restart, pm2 list show no processes and pm2 restart does nothing. This happens even though from what I can tell, the dump.pm2 file in ~/.pm2 doesn't get changed in any way. I saved a copy of a file that was fine and a file that was no good (after crash), and they were binary same. Need to dig into this some more - I can't seem to change my dump.pm2 file to be read only. Probably because I'm a debian idiot. As it stands, if I write some bad node.js code, rebooting the server won't restart what had been running. This is keeping me from being able to migrate servers - Any tips would be helpful. I've been running pm2 for over a year on another server just fine. Maybe wasn't wheezy tho.. |
I changed my dump.pm2 file to be immutable (essentially read only even for root) as suggested by @frankized, and that seems to have helped. Actually got a restart after an out of memory error.
I need a reliable way to create out of memory issues so I can test and make sure this is going to settle things. |
It should work better with PM2 v0.12.13 |
@jshkurti I still have this issue running the latest version of PM2 (0.12.15) on Ubuntu 15. After starting a process and running Please reopen this issue and have a look if you could. Thank you. PM2 0.12.15 |
@buunguyen Are you using Keymetrics ? |
No, I haven't installed Keymetrics on this particular host yet but I intend to. If I install the agent now, do you think it would fix the problem? |
@buunguyen I got the same issue and updating to 0.12.16 seems working now. |
@crashbell thanks. Currently I'm working around by making dump file immutable. A bit annoying but not too painful. Will try the new version later. |
I'm using v0.14.5 and still have the issue. On every reboot, the .pm2/dump.pm2 gets cleared. |
doesn't work with vanilla instructions on ubuntu 14.04 either (npm install pm2) and startup steps and save. |
Same problem on RedHat 7.0 :( the dump file is empty after reboot |
I also got the same problem on ubuntu 14.04 and PM2 0.14.7 |
My server restarted last night and then all processes were started automatically. The dump file wasn't empty and Keymetrics was installed. I don't remember if I did any configuration. I'll check around and let you know soon. Still using 0.12.16 |
Likewise, I have the same issue - a reboot wipes out ~/.pm2/dump.pm2. I am on a BeagleBone Black running Ubuntu with pm2 0.14.7. @tangjie0710 My start-up script does have PM2_HOME set: |
I've has similar problems, and checked other online threads. |
Thanks @glskiba ! |
I love it when I google a problem I'm having and I get back to a GitHub issue thread that I don't even remember commenting on. Lost a process after a reboot earlier this week. Couldn't write to the dump.pm2 file because I had set it to immutable 6 months ago. I don't remember doing that specifically, but there it is in my comment. For the record, updating PM2 to the latest version seems to have solved my reboot problem from 6 months ago - Now all my processes restart. Debian server working just fine. Apologies for using this thread as my dev log. But maybe this helps someone else. |
I did some narrowing of the same problem that I have now. Ubuntu 15.04 configured pm2 with:
edited pm2-init.sh file:
after reboot dump file is being emptied pm2.log file contains then (this is only what has been written to the log after executing
|
Debian GNU/Linux 8 (jessie) / systemd / io.js 2.5.0 / pm2 0.14.7 Same issue: after a reboot, the dump.pm2 contains an empty array (i.e. I had to resort to @frankized 's solution (thank you!) which is an easy workaround for my small project. |
I am not sure why, but for me pm2 is totally ignoring user configs. what I need to do is : But I get error: How is it that even if I run command as a different user, it still goes into the original user's home? I tried the startup script changing user to Also when I run |
Same error for me in Raspbian on Raspberry pi 2. Tried to This is a crucial feature. |
I am also surprised to see the trouble with this, so far in to the life of a generally very slick and polished tools - it seems to be right at the heart of pm2's purpose. |
The workaround has been already implemented: f2714c7 |
@kylecordes I agree and I've given up on the pm2 save functionality. The workarounds don't seem to work for me reliably on the Amazon Linux AMIs. My solution was to create a bash script that runs the "pm2 start server.js" command and then create another bash script as a service in /etc/init.d/myserver. I can then start, stop, and restart pm2 with the command "service myserver start" My pm2 start/stop/restart scripts look something like (this is my avoneu_start.sh):
I then create a file in /etc/init.d/ called avoneu. My service bash script looks like:
Also, I can easily ensure the the service auto starts on reboots by running the command
This solution has been 100% reliable for the last 8 months and I was able to make it part of my server formation templates. I don't fault pm2 for having auto start issues; auto start on each Linux OS can be very different and it is hard to account for every minor difference. |
I am using Amazon Linux. I found that it worked for me if I did the chattr thing to forcibly prevent the file from being damaged. The version I'm using already had the "save" command in the script commented out, that was not enough to prevent the breakage. So apparently it is not just that one command causing the breakage, there is something beyond that going on. |
So what is the final state with this? If this is the case, there is still a bug, which is: when you don't save anything and just power off the machine (no reboot or shutdown command, direct power off) and restart the final state is there (even though pm2 save command was not run before!) So some things apparently get saved. I just do not understand why it just fails to work on machine reboots. |
I'm using Microsoft Azure, install pm2 with UPDATE: I think it caused by So, just check your default shell, does it work with |
I had to make the file immutable also, on Ubuntu 15.10. The dump file is being cleared when I do "pm2 kill". Apparently there is some problem... |
Problem still exists. Behavior suggests a race condition. Sometimes apps are restarted after reboot, sometimes not. When the dump file is wiped out, at least there is a reason for the failure to restart apps. But it also sometimes happens that the dump file is still intact after reboot, but apps are not restarted. Smells like race condition(s) to me, but I have no proof. These are multiple, successive reboots. There's no other change in system configuration between reboots. This is disappointing, but fortunately there are lots of other ways to run jobs at startup on a linux box if pm2 ends up not being reliable on my boxes. Making the dump file immutable may solve the problem for some systems. But immutability does not solve the problem for my minimal Debian 8.5 boxes. I made the dump file immutable, so at least it "can't" be rewritten. (And I have cached a copy and verify that it is not changing between reboots.) Then I do a series of reboots and checked to see if my app restarted: reboot 1 ... app running So even after the file is made immutable, the behavior is inconsistent. I think it's fair to call that a surprising result. I suspect a race condition. As others have reported, the logs reflect the problem but offer no clues to the cause. Not that it should matter but this experiment was performed on a VM. All |
Just to throw the solution I found into this massive issue... However, this is what was generated at
So the problem is that |
@boatrite Thanks for that link and example. That addresses one of the several behaviors which make RCA challenging for this constellation of related behaviors:
I like the goals of pm2 and wish I had time to dive into the code. Right now I need to go with the reliability and transparency of init/systemd. Even though our clusters are small and well instrumented, we'll keep an eye on pm2. Thanks! |
Having read this commentary. I have tried everything which was plausible. The dump file was not getting overwritten (wiped out) on reboot, but the dump file structure was changed from human readable to one single line. By making it read only this did save the human readable structure. The pm2-init.sh file already had the super dump line already commented out. On reboot pm2 does not run as a service, but it does run demonized when I try pm2 list. In any case the app does not survive reboot on ubuntu 14.04. Any other tips and tricks I should try? |
New startup system that will surely fix a bunch of problems mentioned on this issue: |
I'm currently having this issue on a raspberry pi 3, with raspbian (latest build) and pm2 v2.4.2 |
@dikkedimi Check this issue : #2747 |
"pm2 resurrect" works for me after rebooting - but not automatically. |
Is there any way to save and resurrect the running state of each? Lets say:
Can I set it up so that fooApp/bazApp are online, and barApp is stopped on reboot? Currently they start up all online. |
Currently is not possible to save and resurrect the exact state. But this is part of a reflection about pm2 save/resurrect, see this issue : #3759 |
Una disculpa por comentar en español en un foro en Inglés pero encontré la solución al problema. 1.- pm2 start "ejemplo.js" Después de ejecutar lo anterior hacer un "sudo reboot" y con eso el problema queda resuelto. Saludos. |
Hi, I currently having this issue on RHEL 8, I am a bit confused because previously I was testing it on CentOS and it was perfectly fine. |
@Ext3R Is this issue fixed yet? According to the official docs, they make it sound like adding the startup script and @wallet77 How can I auto resurrect all saved processes without running any cronjob for |
|
por el momento no, amigo, PM2 lo utilicé hace 2 años para realizar un proyecto en raspberry :( |
same issue:
pm2 version is 5.3.1 so far pm2 seems to be quite abandoned to me, neither autostart nor ability to monitor logs live with "pm2 monit" features are working. |
happends again in bun beta @Unitech |
I am running centos7 and have just installed PM2.
Added a service "pm2 start app.js --name lets-chat" which starts the program and its working fine.
But after running pm2 startup centos and the pm2 save, upon reboot there is nothing list when I run pm2 list.
I'm not sure I am doing something wrong or if there is a bug.
Thanks
The text was updated successfully, but these errors were encountered: