WIP: ansible: add ansible tower playbook

[gdams]

This playbook will create the basic AWX tower machine. There are a few things that we will probably still want to add:

1. Create a github app and auto configure the github auth CC @rvagg (we'll need one to be created).
2. Auto create the job templates so that we don't have to manually create each job template from a playbook.
3. Add a DNS entry for the machine and enable SSL using Cloudflare.

[gdams]

@maclover7 I'm happy to move it over if others from @nodejs/build agree?

[mhdawson]

Moving to ansible-tower role sounds good to me.

[gdams]

@rvagg could you add this machine to the cloudlare DNS record so that I can configure the job to point to ansible.nodejs.org. Ideally we want to be using the cloudflare front end ssl as well.

[gdams]

@maclover7 I've now moved this to an ansible-tower role

[rvagg]

DNS entry done. Although I've had to change SSL to "flexible" to allow the backend to be http rather than https. Apparently in "full", as it was set, we could even expose https on the server with a self-signed certificate and it would be cool with that. Perhaps we should do that so the Cloudflare<->host comms are encrypted as well? We have a wildcard certificate but that should go away soon if we switch to letsencrypt or we might just switch to Cloudflare's own certificates. Or, maybe we could do letsencrypt on this ansible host for comms with Cloudflare? Can we expose /.well-known/ on it to verify with letsencrypt?

So next up, authenticating with GitHub. There is an OAuth app, like our Jenkins servers, and there's GitHub apps. It looks like we might need the latter so it can access GitHub resources? I'm not really sure how that works and what it can access and as who, but this is the config screen:

Are there docs somewhere about this we can see for Ansible Tower to understand what the implications are? We'll probably need to give the TSC a heads-up.

[rvagg]

I had to switch back to "full" ssl on Cloudflare because it screwed up nodejs.org! I don't know how or why but it made it stick clients into a redirect loop.

So for now, we're going to need an ssl cert of some kind on this new server for it to work via https.

[rvagg]

@gdams sorry for the complexity but I don't see an easier way around this for now, our options I think are:

1. Use letsencrypt on this server—we don't have Ansible scripts we can copy for this except for some config file changes and doc updates for manual bits for dist.libuv.org on the main web server, see d16509c. This would allow us to either expose the server directly or put it through Cloudflare.
2. Generate a self-signed certificate for https on this server. Should be more straightforward than setting up letsencrypt with Ansible I think. If we do this then we should only expose it via Cloudflare which won't care about self-signed.

Since you're probably going to be stuck with the work I'll leave it up to you. I have no preference.

[gdams]

@rvagg I'm happy to use letsencrypt! Looks like there is already a nice ansible module that we could implement... https://docs.ansible.com/ansible/2.5/modules/letsencrypt_module.html

[sam-github]

This appears to be abandoned, but should it be picked up again? Its not clear.

@mhdawson @rvagg, close as not-in-progress, label as needs-help? Other?