Support for Pascal architecture GPUs

[DwayneX77]

Wonderful work presented! May I ask if there is any plan to expand support list to include Pascal architecture GPUs? Take Tesla P-Series as examples.

[tridao]

We currently support Turing (e.g. RTX 2080) and Ampere (e.g. RTX 3080) GPUs. We rely on tensor cores for matrix multiplication, which older GPUs lack.
You might be interested in the memory-efficient attention implemented by the xformers team (targeting fp32 instead of fp16), which I believe supports Pascal cards:
facebookresearch/xformers#267
facebookresearch/xformers#281

[DwayneX77]

Thanks! Your advice helps a lot.

We currently support Turing (e.g. RTX 2080) and Ampere (e.g. RTX 3080) GPUs. We rely on tensor cores for matrix multiplication, which older GPUs lack. You might be interested in the memory-efficient attention implemented by the xformers team (targeting fp32 instead of fp16), which I believe supports Pascal cards: facebookresearch/xformers#267 facebookresearch/xformers#281

[fmassa]

Hi,

We have just pushed a PR in facebookresearch/xformers#362 which contains V100 and P100 support as well, and dispatches to FlashAttention for the cases where it is supported.

[ifffrt]

Hi,

We have just pushed a PR in facebookresearch/xformers#362 which contains V100 and P100 support as well, and dispatches to FlashAttention for the cases where it is supported.

Would this PR make support for Maxwell GPUs possible too?