Option to change CoreDump directory / disable CoreDump globally

[clungzta]

My device has limited storage on the boot disk (it is an embedded ARM device).

The rosmon coredumps to tmp ("/tmp/rosmon-node-XXXXXX") are well over 1GB in my case, and have just filled my boot drive.

I've found that the "enable-coredumps" attribute is available for each node. However, there is no global parameter to disable all coredumps, or change the dump directory (e.g. to a larger external disk).

It may be worthwhile having such global parameters. I could create PR if suitable.

[xqms]

You are correct - currently there is no way to disable coredumps via a rosmon option or to change the directory.

I can suggest two workarounds for your case:

1. Set the system core dump size limit to something small, but non-zero:

   $ ulimit -c 1
   $ mon launch ...

   This will prevent coredumps, since rosmon only enables coredumps if the limit is zero or unlimited:
   

   rosmon/rosmon_core/src/monitor/shim.cpp

   Lines 121 to 130 in abdae1f

   	rlimit limit;
   	if(getrlimit(RLIMIT_CORE, &limit) == 0)
   	{
   	// only modify the limit if coredumps are disabled entirely
   	if(limit.rlim_cur == 0)
   	{
   	limit.rlim_cur = limit.rlim_max;
   	setrlimit(RLIMIT_CORE, &limit);
   	}
   	}

2. Just symlink (or mount --bind) your bigger external harddrive to /tmp.

A PR for a global option would be welcome (both for disabling entirely and changing the location)! I would also suggest supporting an environment variable (maybe ROSMON_DISABLE_COREDUMPS and ROSMON_PROCESS_DIR), that is easier to set system-wide.

[romainreignier]

As a workaround, I would suggest to install systemd-coredump. It will grab and store all coredumps so rosmon does not create one in /tmp, see rosmon output:

 [rosmon]: Stopping node my_node
my_node: terminate called after throwing an instance of 'class_loader::LibraryUnloadException'
my_node:   what():  Attempt to unload library that class_loader is unaware of.
my_node: my_node died from signal 6
my_node: my_node left a core dump
my_node: Determined pattern '/tmp/rosmon-node-a2RzFF/core'
my_node: Could not find a matching core file :-(

But I can find it with:

$ coredumpctl -1
TIME                            PID   UID   GID SIG COREFILE  EXE
Thu 2020-05-14 16:54:55 CEST  15088  1000  1000   6 present   /home/romain/ros_ws/devel/.private/my_package/lib/my_package/my_node

And then the max size used by coredumps can be chosen by the MaxUse in /etc/systemd/coredump.conf.

[xqms]

@romainreignier Interesting, I wasn't aware of systemd-coredump. I'll just point out that if you go down this route, you will lose rosmon's builtin debugging features (i.e. just selecting a crashed node and hitting d to open the debugger). But it's obvious that systemd-coredump is better suited for situations where core dump storage has to be limited.

I guess the proper thing would be to integrate support for systemd-coredump and query it for the coredump location. Then you get the best of both worlds...

[romainreignier]

Indeed, it would be nice :)

[xqms]

The integration with systemd-coredump was done in #125 - it was surprisingly easy :)

#125 also changes the default behavior (without systemd-coredump) to always delete the second-to-last coredump. This should help prevent core dump pileup when a node is frequently crashing. Users who want explicit control over coredump storage (and also benefit from coredump compression) should install systemd-coredump. I placed a hint for that into the installation instructions on the ROS wiki.

I'll close this issue - if either of you have additional remarks, feel free to reopen.