[FEA] Qualification tool: Improve memory consumption while processing large eventlogs.

[nartal1]

Is your feature request related to a problem? Please describe.
This is followon from discussion NVIDIA/spark-rapids#3714 (comment) . Currently if the eventlogs are large and exceeds heap size we throw OOM error and exit the program.
Currently we have documented it to increase the heap size(-Xmx8G) to avoid OOM errors.

Describe the solution you'd like
Would like to improve memory consumption so that large event log files can be read without throwing OOM error.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Current OOM error:

21/09/30 15:15:32 ERROR Profiler: OOM error while processing large file file:/home/nartal/CPU_runs/application_1630450374626_0001_1.Increase heap size.
java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.util.Arrays.copyOf(Arrays.java:3332)
	at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:596)
	at java.lang.StringBuilder.append(StringBuilder.java:190)
	at java.io.BufferedReader.readLine(BufferedReader.java:358)

Tasks

Preview Give feedback

1. [TASK] Optimize the storage of accumulables in core tools #1202
   bug core_tools +2
   
2. [TASK] Create a stub to store Spark StageInfo in core tools #1206
   bug core_tools +2
   
3. [TASK] Optimize the storage used by SQLAccumProfileResults #1272
   core_tools +1
   
4. [TASK] Propagate using of reference in dumping core output
5. [TASK] Optimize the map between execs and stageIDs