version 2.0.0 reporting resources updated #138

jeremymv2 · 2016-11-01T19:35:33Z

Cookbook version

2.0.0

Chef-client version

12.15.19

Platform Details

ubuntu 14.04

Scenario:

version 2.0.0 uses chef-handler in attempt to not report resources being updated, however each subsequent chef-client converge is reporting resources updated.

Steps to Reproduce:

use wrapper cookbook with these attributes:

default['audit']['inspec_version'] = '1.2.0'

# collector possible values: chef-server, chef-compliance, chef-visibility, json-file
# chef-visibility requires inspec version 0.27.1 or above
default['audit']['collector'] = 'chef-server'

# Attributes server, insecure and token/refresh_token are only needed for the 'chef-compliance' collector
# server format example: 'https://comp-server.example.com/api'
default['audit']['server'] = nil

# choose between the permanent refresh_token or ephemeral token(access_token). Needed only for the 'chef-compliance' collector
default['audit']['refresh_token'] = nil

# the token(access_token) expires in 12h after creation
default['audit']['token'] = nil

# set this insecure attribute to true if the compliance server / chef server uses self-signed ssl certificates
default['audit']['insecure'] = nil

# Chef Compliance organization to post the report to. Defaults to Chef Server org if not defined
# needed for the 'chef-compliance' collector, optional for 'chef-server' collector
default['audit']['owner'] = nil

# raise exception if Compliance API endpoint is unreachable
# while fetching profiles or posting report
default['audit']['raise_if_unreachable'] = true

# fail converge if downloaded profile is not present
default['audit']['fail_if_not_present'] = false

# by default run audit every time
default['audit']['interval']['enabled'] = false

# by default run compliance once a day
default['audit']['interval']['time'] = 1440

# quiet mode, on by default because this is testing, resources aren't converged in the normal chef sense
default['audit']['quiet'] = true

# overwrite existing profile in upload mode
default['audit']['overwrite'] = true

# use json format since this is for reporting
default['audit']['format'] = 'json'

# set profiles to empty array as default
default['audit']['profiles'] = [{
      'name' => 'linux',
      'compliance' => 'base/linux'
}]

Expected Result:

Chef-client runs should report 0/x resources updated at the end of the report handlers phase.

Actual Result:

I'm seeing 2 resources updated on each chef-client converge.

root@node:/tmp/vagrant-chef# chef-client -c client.rb
[2016-11-01T19:32:37+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.15.19
[2016-11-01T19:32:37+00:00] INFO: *** Chef 12.15.19 ***
[2016-11-01T19:32:37+00:00] INFO: Platform: x86_64-linux
[2016-11-01T19:32:37+00:00] INFO: Chef-client pid: 4712
[2016-11-01T19:32:39+00:00] INFO: Run List is [recipe[audit_wrapper]]
[2016-11-01T19:32:39+00:00] INFO: Run List expands to [audit_wrapper]
[2016-11-01T19:32:39+00:00] INFO: Starting Chef Run for node
[2016-11-01T19:32:39+00:00] INFO: Running start handlers
[2016-11-01T19:32:39+00:00] INFO: Start handlers complete.
[2016-11-01T19:32:39+00:00] INFO: HTTP Request Returned 404 Not Found:
resolving cookbooks for run list: ["audit_wrapper"]
[2016-11-01T19:32:39+00:00] INFO: Loading cookbooks [audit_wrapper@0.1.0, audit@2.0.0, compat_resource@12.16.1, chef_handler@2.0.0]
Synchronizing Cookbooks:
  - audit_wrapper (0.1.0)
  - audit (2.0.0)
  - compat_resource (12.16.1)
  - chef_handler (2.0.0)
Installing Cookbook Gems:
Compiling Cookbooks...
[2016-11-01T19:32:39+00:00] INFO: Chef Handlers will be located at: /var/chef/handlers
Recipe: chef_handler::default
  * remote_directory[/var/chef/handlers] action create
  Recipe: <Dynamically Defined Resource>
    * cookbook_file[/var/chef/handlers/README] action create (up to date)
     (up to date)
  Converging 5 resources
Recipe: chef_handler::default
  * remote_directory[/var/chef/handlers] action nothing (skipped due to action :nothing)
Recipe: audit::default
  * inspec[inspec] action install
    * chef_gem[inspec] action install (up to date)
    - install/update inspec[2016-11-01T19:32:39+00:00] WARN: Using inspec version: (1.2.0)

    - verifies the inspec version
    * chef_gem[inspec] action install (up to date)

  * directory[/var/chef/cache/handler] action create (up to date)
  * cookbook_file[/var/chef/cache/handler/audit_report.rb] action create (up to date)
  * chef_handler[Chef::Handler::AuditReport] action enable[2016-11-01T19:32:39+00:00] INFO: Disabling Chef::Handler::AuditReport as a report handler.

    - disable Chef::Handler::AuditReport as a report handler[2016-11-01T19:32:39+00:00] INFO: Disabling Chef::Handler::AuditReport as a exception handler.

    - disable Chef::Handler::AuditReport as a exception handler
    - load Chef::Handler::AuditReport from /var/chef/cache/handler/audit_report.rb[2016-11-01T19:32:39+00:00] INFO: Enabling Chef::Handler::AuditReport as a report handler.

    - enable chef_handler[Chef::Handler::AuditReport] as a report handler[2016-11-01T19:32:39+00:00] INFO: Enabling Chef::Handler::AuditReport as a exception handler.

    - enable chef_handler[Chef::Handler::AuditReport] as a exception handler
[2016-11-01T19:32:39+00:00] INFO: Chef Run complete in 0.674548823 seconds

Running handlers:
[2016-11-01T19:32:39+00:00] INFO: Running report handlers
[2016-11-01T19:32:39+00:00] WARN: Format is json-min
[2016-11-01T19:32:39+00:00] INFO: Initialize InSpec
[2016-11-01T19:32:39+00:00] INFO: Running tests from: [{:name=>"linux", :compliance=>"base/linux"}]
[2016-11-01T19:32:40+00:00] INFO: Reporting to chef-server
[2016-11-01T19:32:40+00:00] INFO: Control Profile: ["linux"]
[2016-11-01T19:32:40+00:00] INFO: Control Profil: linux
[2016-11-01T19:32:40+00:00] INFO: Compliance Profils: [{:owner=>"base", :profile_id=>"linux"}]
[2016-11-01T19:32:40+00:00] INFO: Report to Chef Server: https://chef-server.test/compliance/organizations/brewinc/inspec
  - Chef::Handler::AuditReport
Running handlers complete
[2016-11-01T19:32:40+00:00] INFO: Report handlers complete
Chef Client finished, 2/9 resources updated in 02 seconds
root@node:/tmp/vagrant-chef#

The text was updated successfully, but these errors were encountered:

jeremymv2 · 2016-11-01T20:51:35Z

It seems to be coming from enabling/disabling report and exception handlers.

vjeffrey · 2016-11-02T01:02:13Z

yup, i confirmed it's

&&&&&&&& updated resources
inspec[inspec]
chef_handler[Chef::Handler::AuditReport]

the original issue that brought up the idea of the handler mentioned that resources were being reported as 'converged'. i'm not super familiar with chef and resources...is resources being reported as converged and resources being reported as updated the same thing? i'm not quite sure how to get around these updated resources

jeremymv2 · 2016-11-02T13:46:47Z

Yes, in this context they are the same thing.

The default report and exception handlers run at the end of the execution phase (converge) and the report handler sends back the updated_resources object which is a list of resources that were marked as updated as a result of the chef-client run.

chris-rock · 2016-11-02T14:17:17Z

@jeremymv2 Any ideas how we can get around that? I was thinking that Chef Handler are covering that properly

jeremymv2 · 2016-11-02T15:26:10Z

Taking a deeper look. Looks like it's the inspec resource and the handler that account for the two updates.

[1] pry(#<Chef::Handler::AuditReport>)> run_status.updated_resources.each {|r| Chef::Log.warn "#{r.to_s}"}
[2016-11-02T15:24:13+00:00] WARN: inspec[inspec]
[2016-11-02T15:24:13+00:00] WARN: chef_handler[Chef::Handler::AuditReport]
=> [<inspec[inspec] @name: "inspec" @noop: nil @before: nil @params: {} @provider: nil @allowed_actions: [:nothing, :install] @action: [:install] @updated: true @updated_by_last_action: true @supports: {} @ignore_failure: false @retries: 0 @retry_delay: 2 @source_line: "/var/chef/cache/cookbooks/audit/recipes/default.rb:4:in `from_file'" @guard_interpreter: nil @default_guard_interpreter: :default @elapsed_time: 0.496733461 @sensitive: false @resource_name: :inspec @declared_type: :inspec @cookbook_name: "audit" @recipe_name: "default" @version: "1.2.0">,
 <chef_handler[Chef::Handler::AuditReport] @name: "Chef::Handler::AuditReport" @noop: nil @before: nil @params: {} @provider: nil @allowed_actions: [:nothing, :enable, :disable] @action: [:enable] @updated: true @updated_by_last_action: true @supports: {:report=>true, :exception=>true} @ignore_failure: false @retries: 0 @retry_delay: 2 @source_line: "/var/chef/cache/cookbooks/audit/recipes/default.rb:19:in `from_file'" @guard_interpreter: nil @default_guard_interpreter: :default @elapsed_time: 0.004970695 @sensitive: false @declared_type: :chef_handler @cookbook_name: "audit" @recipe_name: "default" @source: "/var/chef/cache/handler/audit_report.rb" @class_name: "Chef::Handler::AuditReport" @arguments: []>]
[2] pry(#<Chef::Handler::AuditReport>)>

chris-rock · 2016-11-02T16:34:58Z

@vjeffrey figured, those events happen:

* chef_handler[Chef::Handler::AuditReport] action enable
   - disable Chef::Handler::AuditReport as a report handler
   - load Chef::Handler::AuditReport from /opt/kitchen/cache/handler/audit_report.rb
   - enable chef_handler[Chef::Handler::AuditReport] as a report handler

Not sure how we can prevent them

jeremymv2 · 2016-11-02T17:35:53Z

It's coming from the converge_by lines here: https://github.com/chef-cookbooks/chef_handler/blob/master/providers/default.rb#L31-L57

chris-rock · 2016-11-02T17:38:07Z

@jeremymv2 so we just remove those and it should be fine?

jeremymv2 · 2016-11-02T17:38:12Z

I think one option may be to replicate what the chef_handler cookbook is doing when loading the AuditReport handler library, but do so in a library in the audit cookbook, without using the LWRP's resources from chef_handler cookbook.

chris-rock · 2016-11-02T17:42:09Z

@jeremymv2 not sure I got that

jeremymv2 · 2016-11-02T17:48:22Z

In theory I think it boils down to adding this to a library file:

require "#{::File.join(Chef::Config[:file_cache_path], 'handler')}/audit_report"
Chef::Config.send('report_handlers') << Chef::Handler::AuditReport.new()
Chef::Config.send('exception_handlers') << Chef::Handler::AuditReport.new()

In essence, that's what's going on here: https://github.com/chef-cookbooks/chef_handler/blob/master/libraries/helpers.rb#L26

jeremymv2 · 2016-11-02T19:19:31Z

So that works.. @chris-rock
I tested adding a library function like this:

  def load_audit_handler
    Chef::Log.info("loading handler...")
    require "#{::File.join(Chef::Config[:file_cache_path], 'handler')}/audit_report"
    Chef::Config.send('report_handlers') << Chef::Handler::AuditReport.new
    Chef::Config.send('exception_handlers') << Chef::Handler::AuditReport.new
  end

Notice no "chef_handler" cookbook is loaded and 0/6 resources were updated below.

root@node:/tmp/vagrant-chef# chef-client -c client.rb
[2016-11-02T19:15:14+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.15.19
[2016-11-02T19:15:14+00:00] INFO: *** Chef 12.15.19 ***
[2016-11-02T19:15:14+00:00] INFO: Platform: x86_64-linux
[2016-11-02T19:15:14+00:00] INFO: Chef-client pid: 8141
[2016-11-02T19:15:15+00:00] INFO: Run List is [recipe[audit_wrapper]]
[2016-11-02T19:15:15+00:00] INFO: Run List expands to [audit_wrapper]
[2016-11-02T19:15:15+00:00] INFO: Starting Chef Run for node
[2016-11-02T19:15:15+00:00] INFO: Running start handlers
[2016-11-02T19:15:15+00:00] INFO: Start handlers complete.
[2016-11-02T19:15:15+00:00] INFO: HTTP Request Returned 404 Not Found:
resolving cookbooks for run list: ["audit_wrapper"]
[2016-11-02T19:15:15+00:00] INFO: Loading cookbooks [audit_wrapper@0.1.0, audit@2.0.0, compat_resource@12.16.1]
Synchronizing Cookbooks:
  - compat_resource (12.16.1)
  - audit (2.0.0)
[2016-11-02T19:15:15+00:00] INFO: Storing updated cookbooks/audit_wrapper/attributes/default.rb in the cache.
  - audit_wrapper (0.1.0)
Installing Cookbook Gems:
Compiling Cookbooks...
Recipe: audit::default
  * chef_gem[inspec] action install (up to date)
  * directory[/var/chef/cache/handler] action create (up to date)
  * cookbook_file[/var/chef/cache/handler/audit_report.rb] action create (up to date)
[2016-11-02T19:15:16+00:00] INFO: loading handler...
  Converging 3 resources
  * chef_gem[inspec] action install (up to date)
  * directory[/var/chef/cache/handler] action nothing (skipped due to action :nothing)
  * cookbook_file[/var/chef/cache/handler/audit_report.rb] action create (up to date)
[2016-11-02T19:15:16+00:00] INFO: Chef Run complete in 0.308372376 seconds

Running handlers:
[2016-11-02T19:15:16+00:00] INFO: Running report handlers

Frame number: 0/21

From: /var/chef/cache/handler/audit_report.rb @ line 14 Chef::Handler::AuditReport#report:

     9:
    10:        require 'pry'
    11:        binding.pry
    12:
    13:         # ensure reporters is array
 => 14:         reporters = handle_reporters(node['audit']['collector'])
    15:
    16:         # collect attribute values
    17:         server = node['audit']['server']
    18:         user = node['audit']['owner']
    19:         token = node['audit']['token']

[1] pry(#<Chef::Handler::AuditReport>)> run_status.updated_resources.each {|r| Chef::Log.info "#{@line_prefix}#{r.to_s}"}
=> []
[2] pry(#<Chef::Handler::AuditReport>)>
[2016-11-02T19:15:19+00:00] WARN: Format is json-min
[2016-11-02T19:15:19+00:00] INFO: Initialize InSpec
[2016-11-02T19:15:19+00:00] INFO: Running tests from: [{:name=>"linux", :compliance=>"base/linux"}]
[2016-11-02T19:15:20+00:00] INFO: Reporting to chef-server
[2016-11-02T19:15:20+00:00] INFO: Control Profile: ["linux"]
[2016-11-02T19:15:20+00:00] INFO: Control Profil: linux
[2016-11-02T19:15:20+00:00] INFO: Compliance Profils: [{:owner=>"base", :profile_id=>"linux"}]
[2016-11-02T19:15:20+00:00] INFO: Report to Chef Server: https://chef-server.test/compliance/organizations/brewinc/inspec
  - Chef::Handler::AuditReport
Running handlers complete
[2016-11-02T19:15:20+00:00] INFO: Report handlers complete
Chef Client finished, 0/6 resources updated in 06 seconds
root@node:/tmp/vagrant-chef#

I'm going to ping @lamont-granquist though to make sure I'm not overlooking anything.

chris-rock · 2016-11-02T19:20:26Z

@jeremymv2 Awesome! Looking forward to have a PR?

jeremymv2 · 2016-11-02T21:48:43Z

Yes - working on changing the inspec resource as well since it was also reporting as updated.

vjeffrey · 2016-11-02T21:52:24Z

thank youuuuu!!!!

chris-rock · 2016-11-03T10:28:41Z

Solved via #142

vjeffrey modified the milestone: 2.0 feedback fixes Nov 1, 2016

jeremymv2 mentioned this issue Nov 2, 2016

fixing resources reporting as updated #142

Merged

4 tasks

chris-rock closed this as completed Nov 3, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

version 2.0.0 reporting resources updated #138

version 2.0.0 reporting resources updated #138

jeremymv2 commented Nov 1, 2016

jeremymv2 commented Nov 1, 2016

vjeffrey commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016

chris-rock commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016

chris-rock commented Nov 2, 2016 •

edited

Loading

jeremymv2 commented Nov 2, 2016

chris-rock commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016 •

edited

Loading

chris-rock commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016 •

edited

Loading

jeremymv2 commented Nov 2, 2016

chris-rock commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016

vjeffrey commented Nov 2, 2016

chris-rock commented Nov 3, 2016

version 2.0.0 reporting resources updated #138

version 2.0.0 reporting resources updated #138

Comments

jeremymv2 commented Nov 1, 2016

Cookbook version

Chef-client version

Platform Details

Scenario:

Steps to Reproduce:

Expected Result:

Actual Result:

jeremymv2 commented Nov 1, 2016

vjeffrey commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016

chris-rock commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016

chris-rock commented Nov 2, 2016 • edited Loading

jeremymv2 commented Nov 2, 2016

chris-rock commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016 • edited Loading

chris-rock commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016 • edited Loading

jeremymv2 commented Nov 2, 2016

chris-rock commented Nov 2, 2016

jeremymv2 commented Nov 2, 2016

vjeffrey commented Nov 2, 2016

chris-rock commented Nov 3, 2016

chris-rock commented Nov 2, 2016 •

edited

Loading

jeremymv2 commented Nov 2, 2016 •

edited

Loading

jeremymv2 commented Nov 2, 2016 •

edited

Loading