Atlantis spamming comments if two PRs are modified in short timespan

[tyler-rt]

Overview of the issue

Atlantis is spamming comments on PRs that do not touch files in a specified project.

the default workspace at path . is currently locked by another command that is running for this pull request.
Wait until the previous command is complete and try again

This appears to occur If two PRs come in within sufficiently little time, and two webhooks are generated, then one of those webhooks will cause this failed plan. In our case, we noticed because our team uses Graphite, and their stack primitive frequently updates 2 or more PRs simultaneously, constantly triggering Atlantis comment spam.

Reproduction Steps

1. Create a mono-repo with structure like

software/file1.py
software/file2.py
infrastructure/atlantis.yaml
infrastructure/project1/main.tf

2. create a atlantis.yaml:

version: 3
projects:
- name: accounts
  dir: infrastructure/project1
  autoplan:
    enabled: true

3. Start atlantis: sudo docker run --restart unless-stopped -d --name atlantis -p 4141:4141 -e ATLANTIS_REPO_CONFIG_JSON='{"repos":[{"id":"/.*/", "repo_config_file":"infrastructure/atlantis.yaml"}]}' ghcr.io/runatlantis/atlantis:v0.27.2

4. Make changes to software/file1.py, and in a separate file make changes to software/file2.py. Create two new PRs at nearly the same time. watch as Atlantis responds with aforementioned comment.

Logs

Example log when Atlantis does not comment:

{"level":"info","ts":"2024-04-26T01:41:30.620Z","caller":"events/working_dir.go:239","msg":"creating dir "/home/atlantis/.atlantis/repos/ReflexTechnologies/codebase/381/default"","json":{}}
{"level":"info","ts":"2024-04-26T01:41:34.611Z","caller":"events/project_command_builder.go:427","msg":"successfully parsed infrastructure/atlantis.yaml file","json":{"repo":"ReflexTechnologies/codebase","pull":"381"}}
{"level":"info","ts":"2024-04-26T01:41:34.612Z","caller":"events/project_command_builder.go:467","msg":"0 projects are to be planned based on their when_modified config","json":{"repo":"ReflexTechnologies/codebase","pull":"381"}}
{"level":"info","ts":"2024-04-26T01:41:34.612Z","caller":"events/plan_command_runner.go:97","msg":"determined there was no project to run plan in","json":{"repo":"ReflexTechnologies/codebase","pull":"381"}}
{"level":"info","ts":"2024-04-26T01:41:34.612Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"381","src":"atlantis/plan","description":"0/0 projects planned successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:41:34.920Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"381","src":"atlantis/policy_check","description":"0/0 projects policies checked successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:41:35.211Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"381","src":"atlantis/apply","description":"0/0 projects applied successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:42:55.244Z","caller":"events/working_dir.go:239","msg":"creating dir "/home/atlantis/.atlantis/repos/ReflexTechnologies/codebase/381/default"","json":{}}
{"level":"info","ts":"2024-04-26T01:42:59.214Z","caller":"events/project_command_builder.go:427","msg":"successfully parsed infrastructure/atlantis.yaml file","json":{"repo":"ReflexTechnologies/codebase","pull":"381"}}
{"level":"info","ts":"2024-04-26T01:42:59.215Z","caller":"events/project_command_builder.go:467","msg":"0 projects are to be planned based on their when_modified config","json":{"repo":"ReflexTechnologies/codebase","pull":"381"}}
{"level":"info","ts":"2024-04-26T01:42:59.215Z","caller":"events/plan_command_runner.go:97","msg":"determined there was no project to run plan in","json":{"repo":"ReflexTechnologies/codebase","pull":"381"}}
{"level":"info","ts":"2024-04-26T01:42:59.215Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"381","src":"atlantis/plan","description":"0/0 projects planned successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:42:59.494Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"381","src":"atlantis/policy_check","description":"0/0 projects policies checked successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:42:59.801Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"381","src":"atlantis/apply","description":"0/0 projects applied successfully.","state":"success","url":""}}

Example log when the undesired comment is triggered

{"level":"info","ts":"2024-04-26T01:51:58.754Z","caller":"events/working_dir.go:239","msg":"creating dir "/home/atlantis/.atlantis/repos/ReflexTechnologies/codebase/373/default"","json":{}}
{"level":"warn","ts":"2024-04-26T01:51:58.940Z","caller":"events/project_command_builder.go:399","msg":"workspace was locked","json":{"repo":"ReflexTechnologies/codebase","pull":"373"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*DefaultProjectCommandBuilder).buildAllCommandsByCfg\n\tgithub.com/runatlantis/atlantis/server/events/project_command_builder.go:399\ngithub.com/runatlantis/atlantis/server/events.(*DefaultProjectCommandBuilder).BuildAutoplanCommands\n\tgithub.com/runatlantis/atlantis/server/events/project_command_builder.go:258\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).BuildAutoplanCommands.func1\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:29\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).buildAndEmitStats\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:71\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).BuildAutoplanCommands\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:26\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).runAutoplan\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:85\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:304\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunAutoplanCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:221"}
{"level":"error","ts":"2024-04-26T01:51:58.940Z","caller":"events/instrumented_project_command_builder.go:75","msg":"Error building auto plan commands: the default workspace at path . is currently locked by another command that is running for this pull request.\nWait until the previous command is complete and try again","json":{},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).buildAndEmitStats\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:75\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandBuilder).BuildAutoplanCommands\n\tgithub.com/runatlantis/atlantis/server/events/instrumented_project_command_builder.go:26\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).runAutoplan\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:85\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:304\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunAutoplanCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:221"}
{"level":"info","ts":"2024-04-26T01:51:58.940Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"373","src":"atlantis/plan","description":"Plan failed.","state":"failed","url":""}}
{"level":"error","ts":"2024-04-26T01:51:59.280Z","caller":"events/pull_updater.go:17","msg":"the default workspace at path . is currently locked by another command that is running for this pull request.\nWait until the previous command is complete and try again","json":{"repo":"ReflexTechnologies/codebase","pull":"373"},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*PullUpdater).updatePull\n\tgithub.com/runatlantis/atlantis/server/events/pull_updater.go:17\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).runAutoplan\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:90\ngithub.com/runatlantis/atlantis/server/events.(*PlanCommandRunner).Run\n\tgithub.com/runatlantis/atlantis/server/events/plan_command_runner.go:304\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommandRunner).RunAutoplanCommand\n\tgithub.com/runatlantis/atlantis/server/events/command_runner.go:221"}
{"level":"info","ts":"2024-04-26T01:52:01.192Z","caller":"events/working_dir.go:239","msg":"creating dir "/home/atlantis/.atlantis/repos/ReflexTechnologies/codebase/377/default"","json":{}}
{"level":"info","ts":"2024-04-26T01:52:02.881Z","caller":"events/project_command_builder.go:427","msg":"successfully parsed infrastructure/atlantis.yaml file","json":{"repo":"ReflexTechnologies/codebase","pull":"373"}}
{"level":"info","ts":"2024-04-26T01:52:02.882Z","caller":"events/project_command_builder.go:467","msg":"0 projects are to be planned based on their when_modified config","json":{"repo":"ReflexTechnologies/codebase","pull":"373"}}
{"level":"info","ts":"2024-04-26T01:52:02.882Z","caller":"events/plan_command_runner.go:97","msg":"determined there was no project to run plan in","json":{"repo":"ReflexTechnologies/codebase","pull":"373"}}
{"level":"info","ts":"2024-04-26T01:52:02.882Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"373","src":"atlantis/plan","description":"0/0 projects planned successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:52:03.197Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"373","src":"atlantis/policy_check","description":"0/0 projects policies checked successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:52:03.524Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"373","src":"atlantis/apply","description":"0/0 projects applied successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:52:05.221Z","caller":"events/project_command_builder.go:427","msg":"successfully parsed infrastructure/atlantis.yaml file","json":{"repo":"ReflexTechnologies/codebase","pull":"377"}}
{"level":"info","ts":"2024-04-26T01:52:05.221Z","caller":"events/project_command_builder.go:467","msg":"0 projects are to be planned based on their when_modified config","json":{"repo":"ReflexTechnologies/codebase","pull":"377"}}
{"level":"info","ts":"2024-04-26T01:52:05.222Z","caller":"events/plan_command_runner.go:97","msg":"determined there was no project to run plan in","json":{"repo":"ReflexTechnologies/codebase","pull":"377"}}
{"level":"info","ts":"2024-04-26T01:52:05.222Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"377","src":"atlantis/plan","description":"0/0 projects planned successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:52:05.559Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"377","src":"atlantis/policy_check","description":"0/0 projects policies checked successfully.","state":"success","url":""}}
{"level":"info","ts":"2024-04-26T01:52:05.914Z","caller":"vcs/instrumented_client.go:236","msg":"updating vcs status","json":{"repository":"ReflexTechnologies/codebase","pull-num":"377","src":"atlantis/apply","description":"0/0 projects applied successfully.","state":"success","url":""}}

Related issues

#2200