Enhance build scripts

[tgravescs]

Is your feature request related to a problem? Please describe.
the build script is fairly hardcoded in the shims it always builds. Add some more options like making the versions configurable, perhaps add option to run tests rather then skip, etc.

Along with this we may need to modify dist/pom.xml profiles to pull in the appropriate versions built by the script.

[gerashegalov]

Some more tasks to consider

• Wrap up Interpolate spark.version.classifier in build.dir [databricks] #3523 to speed up multi-shim builds
• Make dist/pom.xml smaller by either rewriting repeating elements of the pom using ant-contrib for loops or by generating the real pom from some template on the fly in a script. Prepare for 3.2.1 shim, various shim build fixes and improvements [databricks] #3923 , [FEA] Roll explicit dependency:copy into a loop in dist/pom.xml #3975
• allow using a consistent unique tag in deploying SNAPSHOT versions of the single-shim builds such that we con force dist to consume a coherent set of poms for the multi-shim dist assembly without fearing stale versions from partially broken builds.
  Check revisions for all shim jars while build all #4266