Allow non-persistent spot requests

[caarlos0]

Where I work we are using terraform to scale our CI workers, with spot instances. The workers will shutdown after some time of inactivity, and the instance shutdown behavior is set to terminate.

But, since terraform forces us to create the spot request as persistent, when we shut the instance down and terminate it, another instance will be launched, so, we can't automatically scale down.

Note that we don't use terraform to manage those instances, just to launch them.

This change should enable us to do that.

I also updated docs.

Thanks!

[jonathanbeber]

+1 👏

[apparentlymart]

Cool!

If I'm understanding this correctly, then requests created with a spot_type of "one-time" will be created by Terraform, and then they can later close of their own accord. terraform refresh (and thus plan) will then see the spot_bid_status change to show it's closed, but no action will be taken other than that. If you wanted to create a new spot instances it would then be necessary to taint the request to force Terraform to create a new one.

For your CI use-case, are you just using Terraform to create a fresh request and then discarding the terraform state and letting the resources clean up themselves?

Does CancelSpotInstanceRequests complete successfully even if the spot request is already closed?

[caarlos0]

@apparentlymart yes, for example, if I terminate the machine created from a spot request of type one-time, the spot request will be set to closed.

So, what I want to do is:

• create a spot request with shutdown behavior set to terminate and with this new property set to one-time
• worker is started up and work for some time
• after some time without nothing to do, a script in the crontab of the worker will shut itself down, thus, closing the spot request

Every time I run terraform for this manner, I remove the tf state file so it would not mess with the last resource.

This way, I can easily scale the workers not spending too much money.

For your CI use-case, are you just using Terraform to create a fresh request and then discarding the terraform state and letting the resources clean up themselves?

Exactly.

Does CancelSpotInstanceRequests complete successfully even if the spot request is already closed?

I don't know, I'm just testing all this stuff right now, so I closed the spot request myself.

[apparentlymart]

Could you test the behavior of CancelSpotInstanceRequests by spinning up a spot instance, letting it terminate as you said, and then running terraform destroy to see if it completes successfully?

Of course this doesn't really matter for your workflow since you're never going to try to destroy a spot request, but it'd be good for it to behave in a consistent manner so that users with other usage patterns don't find themselves stuck with an un-destroyable resource if their instance happens to terminate before they destroy it with Terraform.

[caarlos0]

@apparentlymart OK, I did two tests:

1. Created a one-time spot request and, while it still alive, execute terraform destroy;
2. Created a one-time spot request, terminated the instance and execute terraform destroy.

Both cases worked as expected.

[apparentlymart]

Awesome! Thanks for indulging me. 😄

[caarlos0]

@apparentlymart haha, no problem 🍻

[caarlos0]

@radeksimko any updates on this?

[apparentlymart]

LGTM 🚢

[apparentlymart]

Merging this resulted in a failed build but it appears to be caused by Azure/azure-sdk-for-go#225 rather than anything in this changeset.

[caarlos0]

@apparentlymart thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.