Fix inconsistency between BACKUP_ITEMS and USE_

[tonyskapunk]

Back in issue #94 when working on the documentation and configuration of the settings, it was clear that there is a inconsistency between the list of BACKUP_ITEMS and the USE_ variables (#94 (comment)), it's required to be more consistent on those settings, likely to deprecate the use of BACKUP_ITEMS or create it in a dynamic way based on the USE_ variables.

[tonyskapunk]

I'm thinking in deprecating BACKUP_ITEMS. recap and recaplog are the two scripts that make use of it.

• recap - Uses this setting for the backupfunctionality, that is, copy the last set of logs from BASEDIR into BASEDIR/backup. This can be generated dynamically regardless of the settings in /etc/recap.conf

• recaplog - Uses this setting for compressing/moving logs daily. This also can be generated dynamically regardless of the settings in /etc/recap.conf.

The implementation of this will happen as part of #155 where recap would allow plugins/modules to be written.

[tonyskapunk]

I have implemented the deprecation of BACKUP_ITEMS in branch tonyskapunk/issue155

Being able to dynamically backup/archive/compress/email is needed in order to add more reports through plugins/modules. That's the reason why it was implemented in there.

Documenting in here the tests performed:

Testing Backup:

2018-05-18 11:36:44-05:00 [INFO] --- Starting recap[15710] ---
2018-05-18 11:36:44-05:00 [INFO] -- bash info: 4 4 19 1 release x86_64-unknown-linux-gnu
2018-05-18 11:36:44-05:00 [INFO] recap[15710]: Created lock file: /var/lock/recap.lock
2018-05-18 11:36:44-05:00 [INFO] Starting check for disk space
2018-05-18 11:36:44-05:00 [INFO] Ended check for disk space
2018-05-18 11:36:44-05:00 [INFO] -- Taking backup, storing reports in /var/log/recap/backups
2018-05-18 11:36:44-05:00 [INFO] Starting backup of reports
2018-05-18 11:36:44-05:00 [INFO] Last run was on: 20180518-113616
2018-05-18 11:36:44-05:00 [INFO] Reports found: [ fdisk mysql netstat ps pstree resources ]
2018-05-18 11:36:44-05:00 [INFO] Ended backup of reports
2018-05-18 11:36:44-05:00 [INFO] recap[15710]: Caught signal - deleting /var/lock/recap.lock
2018-05-18 11:36:44-05:00 [INFO] Execution time: 0s
2018-05-18 11:36:44-05:00 [INFO] --- Ending recap[15710] ---

See it backed up fdisk, mysql, netstat, ps, pstree and resources

Re-running recap, disabling fdisk, mysql reports/logs, then backing up again:

2018-05-18 11:38:33-05:00 [INFO] --- Starting recap[15964] ---
2018-05-18 11:38:33-05:00 [INFO] -- bash info: 4 4 19 1 release x86_64-unknown-linux-gnu
2018-05-18 11:38:33-05:00 [INFO] recap[15964]: Created lock file: /var/lock/recap.lock
2018-05-18 11:38:33-05:00 [INFO] Starting check for disk space
2018-05-18 11:38:33-05:00 [INFO] Ended check for disk space
2018-05-18 11:38:33-05:00 [INFO] -- Taking backup, storing reports in /var/log/recap/backups
2018-05-18 11:38:33-05:00 [INFO] Starting backup of reports
2018-05-18 11:38:33-05:00 [INFO] Last run was on: 20180518-113812
2018-05-18 11:38:33-05:00 [INFO] Reports found: [ netstat ps pstree resources ]
2018-05-18 11:38:33-05:00 [INFO] Ended backup of reports
2018-05-18 11:38:33-05:00 [INFO] recap[15964]: Caught signal - deleting /var/lock/recap.lock
2018-05-18 11:38:33-05:00 [INFO] Execution time: 0s
2018-05-18 11:38:33-05:00 [INFO] --- Ending recap[15964] ---

Working as intended, i.e. dynamically finding the right reports/logs to act upon.

Testing sending mail functionality

Using the previous config:

...
2018-05-18 11:49:45-05:00 [INFO] -- Report suffix: 20180518-114945
...
2018-05-18 11:49:56-05:00 [INFO] Starting email report
2018-05-18 11:49:56-05:00 [INFO] Finding reports from this run: 20180518-114945
2018-05-18 11:49:56-05:00 [INFO] Reports found: [ netstat ps pstree resources ]
2018-05-18 11:49:56-05:00 [INFO] Ended email report
2018-05-18 11:49:56-05:00 [INFO] Sending email report to: tony.garcia@rackspace.com
2018-05-18 11:49:56-05:00 [INFO] Sent email report
2018-05-18 11:49:56-05:00 [INFO] recap[17141]: Caught signal - deleting /var/lock/recap.lock
2018-05-18 11:49:56-05:00 [INFO] Execution time: 11s
2018-05-18 11:49:56-05:00 [INFO] --- Ending recap[17141] ---

See it emailed up netsta, ps, pstree, and `resources reports/logs.

Re-running recap, adding back fdisk, mysql reports:

...
2018-05-18 11:50:52-05:00 [INFO] -- Report suffix: 20180518-115052
...
2018-05-18 11:51:03-05:00 [INFO] Starting email report
2018-05-18 11:51:03-05:00 [INFO] Finding reports from this run: 20180518-115052
2018-05-18 11:51:03-05:00 [INFO] Reports found: [ fdisk mysql netstat ps pstree resources ]
2018-05-18 11:51:03-05:00 [INFO] Ended email report
2018-05-18 11:51:03-05:00 [INFO] Sending email report to: tony.garcia@rackspace.com
2018-05-18 11:51:03-05:00 [INFO] Sent email report
2018-05-18 11:51:03-05:00 [INFO] recap[17342]: Caught signal - deleting /var/lock/recap.lock
2018-05-18 11:51:03-05:00 [INFO] Execution time: 11s
2018-05-18 11:51:03-05:00 [INFO] --- Ending recap[17342] --

Similar as the first test, this is working as intended, dynamically finding the

Testing recaplog

The reports/logs for the day before(YESTERDAYDATE) are mysql, netstat, ps and resources:

2018-05-18 12:27:50-05:00 [INFO] --- Starting recaplog[20400] ---
2018-05-18 12:27:50-05:00 [INFO] recaplog (20400): Created lock file: /var/lock/recaplog.lock
2018-05-18 12:27:50-05:00 [INFO] Compressing old log files
2018-05-18 12:27:50-05:00 [INFO] Finding reports from: 20180517
2018-05-18 12:27:50-05:00 [INFO] Reports found: [ mysql netstat ps resources ]
2018-05-18 12:27:50-05:00 [INFO] Packing mysql...
2018-05-18 12:27:50-05:00 [INFO] Moving 21 logs to: /var/log/recap.test/mysql_daily_20180517
2018-05-18 12:27:50-05:00 [INFO] Compressing 21 logs into: /var/log/recap.test/mysql_daily_20180517.log.tar.gz
2018-05-18 12:27:50-05:00 [INFO] Deleting 21 logs.
2018-05-18 12:27:50-05:00 [INFO] Packing netstat...
2018-05-18 12:27:50-05:00 [INFO] Moving 21 logs to: /var/log/recap.test/netstat_daily_20180517
2018-05-18 12:27:50-05:00 [INFO] Compressing 21 logs into: /var/log/recap.test/netstat_daily_20180517.log.tar.gz
2018-05-18 12:27:50-05:00 [INFO] Deleting 21 logs.
2018-05-18 12:27:50-05:00 [INFO] Packing ps...
2018-05-18 12:27:50-05:00 [INFO] Moving 22 logs to: /var/log/recap.test/ps_daily_20180517
2018-05-18 12:27:50-05:00 [INFO] Compressing 22 logs into: /var/log/recap.test/ps_daily_20180517.log.tar.gz
2018-05-18 12:27:50-05:00 [INFO] Deleting 22 logs.
2018-05-18 12:27:50-05:00 [INFO] Packing resources...
2018-05-18 12:27:50-05:00 [INFO] Moving 22 logs to: /var/log/recap.test/resources_daily_20180517
2018-05-18 12:27:50-05:00 [INFO] Compressing 22 logs into: /var/log/recap.test/resources_daily_20180517.log.tar.gz
2018-05-18 12:27:50-05:00 [INFO] Deleting 22 logs.
2018-05-18 12:27:50-05:00 [INFO] Deleting log files older than 15 days...
2018-05-18 12:27:50-05:00 [INFO] Deleting: 12218 log files.
2018-05-18 12:27:59-05:00 [INFO] Deleting: 0 empty directories.
2018-05-18 12:27:59-05:00 [INFO] recaplog (20400): Caught signal - deleting /var/lock/recaplog.lock
2018-05-18 12:27:59-05:00 [INFO] --- Ending recaplog[20400] ---

[tonyskapunk]

Closing this one since it has been merged into development, yay!