Should backend_cache default to true?

[andyjarvis]

As of Inspec v2.1.0 the cli option --backend-cache defaults to enabled, yet it appears that in kitchen-inspec the default for backend_cache is disabled. Shouldn't it default to enabled?

Tested with:

chef --version
Chef Development Kit Version: 3.2.30
chef-client version: 14.4.56
delivery version: master (6862f27aba89109a9630f0b6c6798efec56b4efe)
berks version: 7.0.6
kitchen version: 1.23.2
inspec version: 2.2.70

Running 180 inspec tests against a windows 2016 instance took 6 minutes without backend_cache specified but only 35 seconds with backend_cache: true

[stevenoneill-jmfamily]

bump?

[clintoncwolfe]

PR #264 made enabling the backend cache the default, but there is an argument for exposing the option while leasing it disabled by default - it is less surprising behavior. #264 (comment)

[myii]

PR #264 made enabling the backend cache the default, but there is an argument for exposing the option while leasing it disabled by default - it is less surprising behavior. #264 (comment)

We just got around to updating the gems across ~100 repos in our organisation. During that process, we found that a couple of our repos had tests that were failing. Immediately noticed that the actual tests that were failing involved some form of retries.

https://github.com/saltstack-formulas/openvpn-formula/blob/2d7ea151d7a7042356ed580f83a7f376ec53b739/test/integration/default/controls/services_spec.rb#L40-L44

    describe 'Initialization' do
      it 'should be completed', retry: 60, retry_wait: 1 do
        expect(file(logfile).content).to include 'Initialization Sequence Completed'
      end
    end

• Using rspec-retry.

https://github.com/saltstack-formulas/rspamd-formula/blob/19d8fba3e4863b326ba1c9f3d5655dcfc87b30f4/test/integration/default/controls/service.rb#L42-L55

    describe port(pn) do
      # It takes a while to the port to be listening, so we add a little sleep
      before do
        30.times do
          unless port(pn).listening?
            puts "Port #{pn} isn't ready, retrying.."
            sleep 1
          end
        end
      end
      it { should be_listening }
      its('protocols') { should include 'tcp' }
      its('addresses') { should include '127.0.0.1' }
    end

• Appears to be based on this workaround:
  • Add a way to retry a test if failing inspec#4495 (comment)

Tracked the issue down to #264 -- realised that the backend cache prevents new changes from being seen, obviously. Fixed both repos by disabling backend_cache in kitchen.yml:

verifier:
  name: inspec
  ...
  # This formula has InSpec controls which involve retrying tests, to allow time,
  # for example, for services to start up fully.  These retries will not pick
  # up these changes if the backend cache is in use, so it's set to `false` here.
  backend_cache: false

Keeping the benefit of backend_cache for the rest of our repos, though.