Override estimated speedups when estimation model is enabled

[parthosa]

Fixes #878. This PR introduces the following changes:

1. Updates the user tools to replace the Estimated GPU Speedup by the speed ups generated from QualX.
   • As a part of this, add a new column Speed Up Estimation Model which stores which estimation model was used for calculation of speed up (XGBoost or Speedup (heuristic approach)).
   • This column should be present even if the xgboost estimation was disabled.
2. Modify the output directory from QualX to xgboost_predictions and output files as per_sql.csv and per_app.csv.
3. Updates the prediction function to use model based on the platform.
4. Introduce debug variable to toggle printing debug information. Default is False

Output

Platform: onprem:

qualification_summary.csv

,App Name,App ID,Speedup Based Recommendation,Estimated GPU Speedup,Estimated GPU Duration,App Duration,Unsupported Operators Stage Duration,Unsupported Operators Stage Duration Percent,Speed Up Estimation Model
0,NDS - query72,application_1686676198636_0002,Recommended,7.04,126117.00,887621.00,0.00,0.00,XGBOOST
1,Databricks Shell,app-20230906000351-0000,Recommended,6.45,788394.00,5088279.00,83674.00,1.64,XGBOOST

Report Summary:
-------------------------  ----
Total applications            2
RAPIDS candidates             2
Overall estimated speedup  6.53
-------------------------  ----

Platform: dataproc

qualification_summary.csv

,App Name,App ID,Speedup Based Recommendation,Estimated GPU Speedup,Estimated GPU Duration,App Duration,Unsupported Operators Stage Duration,Unsupported Operators Stage Duration Percent,Speed Up Estimation Model
0,NDS - query72,application_1686676198636_0002,Strongly Recommended,4.92,180511.00,887621.00,0.00,0.00,XGBOOST
1,Databricks Shell,app-20230906000351-0000,Recommended,3.91,1302505.00,5088279.00,83674.00,1.64,XGBOOST

Report Summary:
-------------------------  ----
Total applications            2
RAPIDS candidates             2
Overall estimated speedup  4.03
-------------------------  ----

Platform: onprem with XG boost estimation disabled

qualification_summary.csv

,App Name,App ID,Speedup Based Recommendation,Estimated GPU Speedup,Estimated GPU Duration,App Duration,Unsupported Operators Stage Duration,Unsupported Operators Stage Duration Percent,Speed Up Estimation Model
0,NDS - query72,application_1686676198636_0002,Strongly Recommended,3.28,270531.04,887621.00,0.00,0.00,SPEEDUPS
1,Databricks Shell,app-20230906000351-0000,Recommended,1.78,2865463.88,5088279.00,83674.00,1.64,SPEEDUPS

Report Summary:
-------------------------  ----
Total applications            2
RAPIDS candidates             2
Overall estimated speedup  1.91
-------------------------  ----

[amahussein]

Thanks @parthosa
Everything looks good for first iteration. I have quick suggestion though.

[cindyyuanjiang]

Thanks @parthosa!

[amahussein]

Thanks @parthosa