[FEA] Create MIG with Cgroups on YARN Dataproc scripts

[tgravescs]

Is your feature request related to a problem? Please describe.
We found a way to make MIG work with Cgroups on YARN without modifications to yarn. Get this running on Dataproc and create the necessary initialization scripts.

[tgravescs]

One way we can get this to work is by making YARN use the major/minor device numbers for the MIG GI access nodes when it denies access via Cgroups. By denying access to the GI, it denies access to all CI's underneath it.
To get this to work we need to have a wrapper nvidia-smi script and use the container-executor.cfg configuration to override the gpu major device number.

1. modify https://github.com/NVIDIA/spark-rapids-examples/blob/branch-22.04/examples/MIG-Support/yarn-unpatched/scripts/mig2gpu.sh to return the GI access node as the GPU minor device id (/proc/driver/nvidia-caps/mig-minors | grep gpu0/gi1/access). ie the script still enumerates all MIG devices like they are normal GPUs with the modified minor device number. Install this script on each node.
2. Update yarn.env.sh and yarn-site.xml per https://github.com/NVIDIA/spark-rapids-examples/tree/branch-22.04/examples/MIG-Support/yarn-unpatched and new option added in step 1
3. On each node, add the gpu.major-device-number = grep nvidia-caps /proc/devices to the container-executor.cfg file. This has to be per node because different nodes could have different numbers.

With the above changes, YARN will internally pass the major and minor devices for mig GI access to the container-executor to add all GPU MIG devices that this container shouldn't have access to the Cgroup Deny list for that container.

To do this on Dataproc we need an option for user to enable MIG with scripts to do it. This requires A100's and MIG to be enabled and then a reboot to happen for it to take affect. The only way I found to do that is using the --metadata=startup-script-url= option and in that script it installs gpu drivers and enables mig, then does a reboot. On reboot that script will get run again and the script has to configure MIG CI instances per user and then do the normal YARN and spark rapids plugin install and initialization.