Don't refresh modules twice per sync or refresh ops

[DmitryKuzmenko]

What does this PR do?

Some of saltutil functions makes grains calculated twice per call. This happens because these functions fire both refresh_modules and refresh_pillar events while refresh_pillar event refreshes modules already by itself. refresh_modules reloads grains so firing both events we are getting double-grains calculation.
This PR avoids calling of refresh_modules in case refresh_pillar is called.

What issues does this PR fix or reference?

Fix #43941

Tests written?

No

Commits signed with GPG?

Yes

[rallytime]

@DmitryKuzmenko That seems like the right approach to me, but there's some related test failures here: https://jenkins.saltstack.com/job/PR/job/salt-pr-linode-ubuntu14-n/21119/

@saltstack/team-core Is there some expected behavior here that I am missing that these tests are looking for duplicate grain values like that?

[cachedout]

I think this is fine but I'm a little curious about the failing test here.

[DmitryKuzmenko]

That tests are surprising me too.. I'll work on it.

[cachedout]

@DmitryKuzmenko Any update here?

[DmitryKuzmenko]

@cachedout I'll probably have time for this on the next week.

[DmitryKuzmenko]

re-run py

[DmitryKuzmenko]

As for me it looks like removal of one unneeded refresh made the test work faster that produced the failures not because of my changes but because this made the test more reliable. I bet the updated grains module test will fail without my fixes on Jenkins.
I'll continue investigate the reason of the failure.

[DmitryKuzmenko]

Ok, here is the problem:
Each time the module function runs it runs in a standalone process so all the in-memory changes are applied to the only the execution process.
Grains changes are shared to the main process through the filesystem.
After update grains on the FS the module function sends an event to the main process to signal it to refresh the modules. This produces the main process to re-read grains file and run following processes with the updated grains data.
Between sending the event and doing the job there is time. And here's the race condition where the append function lies because it's executed earlier than the previous setvalue(grains_key, []) function is applied to the main process' data.

Tomorrow I plan to think more on the fix an provide a proposal.

[rallytime]

@DmitryKuzmenko Any luck on this one yet?

[DmitryKuzmenko]

@rallytime working on it. I've wrote "tomorrow" but it was my last working day before vacation =))) Sorry for confusion

[cachedout]

@DmitryKuzmenko When you're ready to come back to this, we'll need a merge conflict resolved, please.

[DmitryKuzmenko]

@cachedout sure.

[cachedout]

@DmitryKuzmenko Any updates here?

[DmitryKuzmenko]

@cachedout currently working on Tornado 5.0. Will done this later.

[damon-atkins]

re-run lint

[Ch3LL]

ping @DmitryKuzmenko any update here? We just branched 2017.7.9 which is the last bug fix release for 2017.7, so if this needs to get into the release it needs to be cleaned up and migrated to the 2017.7.9 release branch. thanks :)

[DmitryKuzmenko]

@Ch3LL This PR looks like a proper solution but it's not completed without fixing the race conditions I've described above. I have no solution for that race condition right now and have no time I can spend on it for now. So I don't believe we have a chance to include it to 2017.7 unless we have some free hands.

[Ch3LL]

no worries. When you do have time to get back to this lets move it to the 2018.3 branch, since there will be no more 2017.7 bug releases after 7.9.

[DmitryKuzmenko]

I've rebased to master. Also cherry-picked commits from #55233.

[dwoz]

@DmitryKuzmenko Linter