-
Notifications
You must be signed in to change notification settings - Fork 17
IM 01 Hypervisor Setup
FYI: This document is sorely out of date and needs to be updated. A major revamp has just been performed, the update to the documentation is pending our next release (and some r&r).
Chef must be run as root. If you already have part of what you need, you can skip to the appropriate environment here:
First install Git:
apt install git
Then, create an SSH key and add that SSH key to your GitHub account. This implies you trust the computer with your GitHub account; e.g., the hard drive is encrypted, permissions on the computer are locked down, and the computer is connected to a non-public network.
Clone the machete-cm repository into /opt/kvm and navigate to that directory that
contains the Git repository. Select the branch you need and kick things off with:
# ./chef-solo-run.sh hv [ssh_port]
Then, configure the bridge:
cd src/files/hv
./configure-kvm-bridge.sh [interface_name] [address] [gateway] [usermode_bridge_name] [range]
If these options are unclear, just run the script without any arguments and it will explain how to use them.
Re-run Chef (sorry; the bridge configuration should be done with Chef but it currently isn't).
Finally, create the blueprint virtual machine that will be used as the KVM base for the other two (test, prod).
./create-virtual-machine.sh
The setup for the virt is the same, but the scripts are different.
Please note that this is the setup for a brand new virtual machine, assuming a complete catastrophe. If you are in an environment that
has already been setup once, you can simply copy the existing blueprint environment. Skip to the subsection
Working with Blueprint.
Do the Chef run:
./chef-solo-run.sh virt
Then, from src/files/blueprint:
./configure-interfaces.sh
Do the Chef run again:
./chef-solo-run.sh virt
You may want to change the SQL Server password.
docker exec -it sqlserver bash
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA
# enter password, then:
> ALTER LOGIN SA WITH PASSWORD="<YourNewStrong!Passw0rd>"
Exit: Ctrl-D (for sqlcmd only) then Ctrl-P Ctrl-Q (for Docker). This will gracefully exit the container without stopping the main process.
You will want to note the IP address for SQL Server:
docker network inspect bridge
You will need it if you are editing the appsettings.json file manually. If that is necessary, replace "sqlserver" in the template with the IP address you just obtained, and the password for the database, and fill in the entries.
This repo does not contain additional information about the appsettings.json file, which is a part of the original Machete repo. For further information on tenancy, see that repo. TODO: LINK
As configured on hv, the blueprint environment occupies the same external and internal IPs as the
test environment. To avoid networking issues, please take the test environment out of commission
before cloning the blueprint environment. If you are setting up a new prod environment from the
blueprint, you will need to both (a) take the test environment out of commission, and (b) update
the IP addresses of the interfaces to be consistent with the DNS values found at aws.amazon.com and
in the /etc/hosts file on hv, or create new entries.
You may wish to familiarise yourself with these virsh commands:
virsh and its subcommands
virt-clone
In order to shut down the test environment, you may simply:
virsh suspend test
To clone the blueprint environment:
new_environment_name="bb-8"
sudo virt-clone --original blueprint --file /var/lib/libvirt/images/${new_environment_name}.qcow2
Now, before spinning the machine up (which we'll call bb-8 for purposes of documentation only), go ahead
and do the following as root (this requires libguestfs-tools to be installed, which should already be
the case):
mkdir -p /mnt/bb-8 && guestmount -d bb-8 -i /mnt/bb-8
nano /mnt/bb-8/etc/default/grub
Make sure the following options are set in the file:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0"
GRUB_TERMINAL="serial console"
(Comment out any other options that have the same name with #).
Because of the current state of the blueprint environment you will then need to boot the machine with the
console attached, edit the boot command, and login directly via the command line. You will also need to
run update-grub prior to executing your first Chef run(s).
guestunmount /mnt/bb-8
virsh start bb-8 --console
Act quickly; at the grub login screen, press e to edit the first command:
GNU GRUB version 2.02~beta3-5+deb9u1
+----------------------------------------------------------------------------+
|*Debian GNU/Linux |
| Advanced options for Debian GNU/Linux |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
+----------------------------------------------------------------------------+
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, `e' to edit the commands
before booting or `c' for a command-line.
The highlighted entry will be executed automatically in 4s.
Press the down arrow until you reach the linux command, and edit it to match the following:
GNU GRUB version 2.02~beta3-5+deb9u1
+----------------------------------------------------------------------------+
| search --no-floppy --fs-uuid --set=root 56db41d1-62d2-44b8-b7c5-\|^
|96315855d366 |
| else |
| search --no-floppy --fs-uuid --set=root 56db41d1-62d2-44b8-b7c5-9\|
|6315855d366 |
| fi |
| echo 'Loading Linux 4.9.0-8-amd64 ...' |
| linux /vmlinuz-4.9.0-8-amd64 root=/dev/mapper/blueprint--vg-\|
|root ro console=tty0 console=ttyS0 |
| echo 'Loading initial ramdisk ...' |
| initrd /initrd.img-4.9.0-8-amd64 |
| |
+----------------------------------------------------------------------------+
Minimum Emacs-like screen editing is supported. TAB lists
completions. Press Ctrl-x or F10 to boot, Ctrl-c or F2 for
a command-line or ESC to discard edits and return to the GRUB menu.
Then, proceed to boot (Ctrl-x). Login using ROOT credentials (Contact the administrators). Then:
update-grub
echo 'bb-8' > /etc/hostname
reboot now
# you'll be logged out...
virsh console bb-8
# login again
You should now be able to create a user and proceed to the next section. You may also wish to add your new
user's username to the sudoers file. An SSH server should already be running on the host; you will have to
add your Yubikey's publickey to the /home/${USER}/.ssh/authorized_keys file (perms: 0644).
Now, navigate to the directory where the scripts reside. It will be in an older version of this repository. We are working on getting a new Blueprint running as capacity permits.
cd /home/celiyah/machete-kvm/src/files
ip=your_public_ip
./configure-interfaces.sh ens2 ens3 ${ip}
If you don't have a public interface (they cost money), you'll need to provide a fake IP, such as 127.0.0.1.
Afterwards, edit /etc/network/interfaces and comment out the public interface. The script will tell you
to verify your SSH connection and then do the following:
service sshd restart
service networking restart
However, you may wish to ip addr del 127.0.0.1/24 dev ens3 first, if you provided a fake IP for the public interface.
At this point, you may not have access to the internet. If that's the case you'll need to define a route. So:
ip r a default via 10.0.0.1
This should get you what you want. Please note that this is not the proper configuration for a web head, which should use its public IP address as the default route so that traffic can flow from the web server to the internet.
You can now proceed to your Chef run. You'll need to make sure your public SSH key is available. Although not a recommended practice, I
usually use sudo -E -s for this purpose. Alternative methods are (1) generating a default SSH key for the root user, or (2)
configuring the Chef repository to be readable by a non-root user.
To start a brand new instance of the Machete app, you're going to have to do the following:
- Provide a list of the tenants (i.e., individual websites);
- Edit the connection strings for each tenant; and,
- Restore the databases, if you are doing disaster recovery.
These instructions are to setup a blank testing environment. If you're doing disaster recovery, skip here instead.
Place this file in the app-secrets Docker volume:
scp appsettings.json hv.initech.is:/home/${USER}
port=[The SSH port of the HV]
ssh hv.initech.is -p${port}
scp appsettings.json bb-8.initech.is:/home/${USER} # this assumes you've added an entry in /etc/hosts
ssh bb-8.initech.is
sudo -E -s
cp appsettings.json /var/lib/docker/volumes/app-secrets/_data
cd /var/lib/docker/volumes/app-secrets/_data
Then:
docker attach machete1
cd /app/certs
#
./machete.sh
The above script is only to launch NginX and Machete together. For just Machete, run dotnet Machete.Web.dll.
That is not the full version of the dotnet CLI. It will only run a binary file.
You should have a working website now.