multiple remote-exec's against the same target overwrite script.sh

[arubis]

I'm using terraform's remote-exec provisioner with inline argument to run a script on my configuration management server whenever a new instance is provisioned. This allows me to pass Terraform variables directly into the script, which is very convenient.

Crucially, in case this wasn't clear, I'm using remote-exec to execute not on the instance the enclosing resource block describes, but on another central server.

This method works well so long as I provision one new instance at a time. However, if multiple instances are provisioned at once, since remote-exec always names its script /tmp/script.sh on the destination server, parallel terraform-remote-exec processes sometimes knock each other's scripts over while executing.

A snippet from my app.tf:

  provisioner "remote-exec" {
    connection {
      user = "ubuntu"
      agent = "true"
      host = "${module.salt_master.public_dns}"
    }
    inline = [
      "until sudo /usr/bin/salt-key ; do sleep 5; echo Waiting for saltstack install to complete ; done",
      "until ssh -i (CREDENTIALS) ubuntu@${self.public_dns} hostname ; do echo Connecting to ${self.public_dns} ; sleep 5s ; done",
      "sudo salt-key -y -d app${count.index}.${var.base_domain}; echo Invalidated old key for app${count.index}.${var.base_domain}",
      "sudo salt-key --gen-keys=app${count.index}.${var.base_domain} && echo Created new key for app${count.index}.${var.base_domain}",
      "sudo cp app${count.index}.${var.base_domain}.pub /etc/salt/pki/master/minions/app${count.index}.${var.base_domain} && echo Installed new key for app${count.index}.${var.base_domain} on master",
      "echo Installing new key for app${count.index}.${var.base_domain} on minion",
      "sudo scp -i (CREDENTIALS app${count.index}.${var.base_domain}.* ubuntu@${self.public_dns}:~/",
      "ssh -i (CREDENTIALS) ${self.public_dns} 'sudo mkdir -p /etc/salt/pki/minion; sudo mv ~/app${count.index}.${var.base_domain}.pub /etc/salt/pki/minion/minion.pub; sudo mv ~/app${count.index}.${var.base_domain}.pem /etc/salt/pki/minion/minion.pem'",
      "sudo rm -f app${count.index}.${var.base_domain}.*",
    ]
  }

I have similar provisioners in TF files for different server roles, and have seen a terraform run with output including:

aws_instance.es (remote-exec): Invalidated old key for app0.MYDOMAIN.com
aws_instance.es (remote-exec): Created new key for app0.MYDOMAIN.com
aws_instance.es (remote-exec): Installed new key for app0.MYDOMAIN.com on master
aws_instance.es (remote-exec): Installing new key for app0.MYDOMAIN.com on minion

...which should be managing a key for 'es0.MYDOMAIN.com' but the script.sh was swapped in mid-run due to another simultaneous provisioner.

Could terraform add entropy to the end of a script name, ie. /tmp/script-0EB5.sh, or a timestamp, or some other way to prevent collision?

[mitchellh]

Fixed in PR #1588. Nice edge case catch! Thanks.

[arubis]

Thanks so much!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.