Verify network connectivity for Android mobile devices

[premun]

Context

We have seen issues where Android devices in Helix lose internet connectivity (e.g. caused by network switch failure, so not an issue with the devices themselves). The problem is that this doesn't manifest right away as most tests don't need internet but when an app that needs it hits the affected device, it fails. This creates random failures which are hard to detect or mitigate.

Goal

We should perform some network test before the work item or maybe just in case of failures and retry the work item to mitigate issues.
We also need to have telemetry about that so that we can be alerted and fix the connectivity issues.

We can use the adb shell ping command which performs ping from the device.

[akoeplinger]

Doing it all the time before every workitem might be overkill, I'd prefer only doing it when the workitem fails (and we need to make sure to retry on a different machine).

[premun]

I agree with the overkill. We can totally do it afterwards, however, Helix currently cannot guarantee it will be a different machine.
We can hope that our alerting will notify FR fast enough and they will take the machine offline.

[premun]

I checked and xharness.exe android adb -- shell ping 8.8.8.8 returns exit code 2 for failed network. I need to verify for success but this should give us an easy way

[premun]

We have decided that failures are not that common and that the check is super fast to do so we can afford to do it every time we see a failure