Which KPI of GPU impact PGStrom perfomance most?

[zhaonaiy]

Greetings,
I have 2 questions about PGstrom:
1, If FP32 performance impacts PGStrom at most, other than Tensor core (INT8) ? or other KPI？
2, How PGStrom arrow files support update from PG tables? Manual work? or any script?

 Thanks in adance. 

[kaigai]

1, If FP32 performance impacts PGStrom at most, other than Tensor core (INT8) ? or other KPI？

Usually, PG-Strom's performance is restricted by the PCI-E bus throughput to load data from NVME-SSD to GPU.
Computational portion is usually minor than i/o.

2, How PGStrom arrow files support update from PG tables? Manual work? or any script?

UPDATE is not supported (and hard to support it because of Apache Arrow file format characteristics).
If you want to append rows on existing apache arrow files, use pg2arrow --append mode.

Best regards,

[zhaonaiy]

Thank you for your reply. It's helpful. Another 2 questions: 1, Even though, I/O impact is more than GPU computation power, I still want to know which part capability of GPU is used by PGStrom engine, FP32, FP64, Tensor Core, or others? Because I need this to determine which GPU card to choose, for example, A2 or A10(fp32: 31Tflops, fp64 ~0) , or A100(FP32: 19.5Tflops, but tensor core like INT8 is more powerful) , which is more suitable? 2, If Apache Arrow files is something like Parquet? If a big file will be splitted into multi parts? If append mode, PGStrom will load all files including appeded ones and combine them together in GPU memory? ***@***.*** From: KaiGai Kohei Date: 2024-05-10 23:54 To: heterodb/pg-strom CC: zhaonaiy; Manual Subject: Re: [heterodb/pg-strom] Which KPI of GPU impact PGStrom perfomance most? (Discussion #767) 1, If FP32 performance impacts PGStrom at most, other than Tensor core (INT8) ? or other KPI？ Usually, PG-Strom's performance is restricted by the PCI-E bus throughput to load data from NVME-SSD to GPU. Computational portion is usually minor than i/o. 2, How PGStrom arrow files support update from PG tables? Manual work? or any script? UPDATE is not supported (and hard to support it because of Apache Arrow file format characteristics). If you want to append rows on existing apache arrow files, use pg2arrow --append mode. Best regards, ― Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[kaigai]

1, Even though, I/O impact is more than GPU computation power, I still want to know which part capability of GPU is used by PGStrom engine, FP32, FP64, Tensor Core, or others? Because I need this to determine which GPU card to choose, for example, A2 or A10(fp32: 31Tflops, fp64 ~0) , or A100(FP32: 19.5Tflops, but tensor core like INT8 is more powerful) , which is more suitable?

I think INT32 is most significant for GPU workloads from the computation perspective.
(Note that PG-Strom does not use TensorCores internally.)
So, if GPU-Direct Storage would be available on the Geforce class, it will be the most cost effective option.
NVIDIA RTX (Quadro in the past) might be a good choice, but we have not evaluated these devices yet.

2, If Apache Arrow files is something like Parquet? If a big file will be splitted into multi parts?

Yes, it is called "Record Batch" in Apache Arrow.
For example, pg2arrow makes record-batch per 128MB in default.

If append mode, PGStrom will load all files including appeded ones and combine them together in GPU memory?

No. PG-Strom loads record-batch of apache-arrow files one-by-one but concurrently.
And, PG-Strom processes the record-batch that was loaded earlier first, then discards the record-batch that has been processed to free up memory for the next process. So, it never combines them.

[zhaonaiy]

Thanks for your reply. Then,
1, If we declare a data type is "Bool", in GPU, it still occupies 4B?
2, For PG-strom, if memory bandwidth is important? That means if GPU requires DDR6 or HBMe.
3, Can PGstrom make load balance across multi-GPU? Or can schedule one GPU per process?

Thank you.

[kaigai]

1, If we declare a data type is "Bool", in GPU, it still occupies 4B?

No. It consumes 1byte+1bit (NULL bitmap) if PostgreSQL heap format, and 1bit + 1bit if Apache Arrow format.
Of course, it shall be expanded to 4B register when Bool datum is referenced in calculation.

2, For PG-strom, if memory bandwidth is important? That means if GPU requires DDR6 or HBMe.

Probably, SQL workloads are not optimal for memory bandwidth, so there are little differences in DDR6 and HBM2.
In our past measurement, A100 and L40s reported similar performance in Star Schema Benchmark.

3, Can PGstrom make load balance across multi-GPU? Or can schedule one GPU per process?

PostgreSQL launches worker processes for large table scan, and PG-Strom adopts one GPU per process model.
Eventually, it tends to use multiple GPUs as if it is load balanced.

[zhaonaiy]

Just to clarify,
1, What do you means about "Eventually"? If we have 2 GPUs, then PGStrom of CURRENT version can make them use in one process, like large table scan and calculation?
2, I have a large table with 2 columns, one column is ID (4B）, the other is Bool (2bits). If there's any compression tech when load this large table (assume 1B rows) into GPU memory? Or it stores as 2Bits (for col2), but at calculation it will be expanded to 4B?
Thanks!

[kaigai]

1, What do you means about "Eventually"? If we have 2 GPUs, then PGStrom of CURRENT version can make them use in one process, like large table scan and calculation?

If PostgreSQL chooses parallel query execution, workloads are distributed to multiple worker processes.
This worker process uses one of the candidates GPUs that can be round-robin.
It looks like multiple-GPUs load balancing.

2, I have a large table with 2 columns, one column is ID (4B）, the other is Bool (2bits). If there's any compression tech when load this large table (assume 1B rows) into GPU memory? Or it stores as 2Bits (for col2), but at calculation it will be expanded to 4B?

It depends on the storage option (but PG-Strom does not support compression technology, because decompression is
usually sequential processes thus not suitable for GPUs).
If you choose PostgreSQL's heap table, at the storage level, one row consumes 24B for header, 4B for ID, and 1B+1B for values.
If you choose Apache Arrow file, at the storage level, one row consumes 4B+1bit for ID, and 2bit + 2bit for values.
Regardless of the storage option, when Bool value is loaded to GPU's register, it shall use 32bit register because it is internally
processed by INT32 cores.