[FRONTEND][BACKEND] Cleanup and re-enable optimization with fp8e4b15 #3521
Conversation
…e4b15 Multiple fixes to allow pipelining to happen when generating a matmul with fp8e4b15 inputs. Also clean useless code in the frontend.
| @@ -149,6 +149,24 @@ SmallVector<Value> packI32(const SmallVector<Value> &inValues, Type srcTy, | |||
| } | |||
| return outValues; | |||
| } | |||
|
|
|||
| int getNumElmenetPerThreads(Type type, const LLVMTypeConverter *typeConverter) { | |||
There was a problem hiding this comment.
Typo: Elmenet -> Element
|
I have a few review comments, please hold on for a sec till I'm out of the interview... |
Thanks Justin, I'll push that for now to unblock my wheel update but please add your comments and I'll send a follow up PR |
| @@ -149,6 +149,24 @@ SmallVector<Value> packI32(const SmallVector<Value> &inValues, Type srcTy, | |||
| } | |||
| return outValues; | |||
| } | |||
|
|
|||
| int getNumElmenetPerThreads(Type type, const LLVMTypeConverter *typeConverter) { | |||
There was a problem hiding this comment.
already addressed
| @@ -149,6 +149,24 @@ SmallVector<Value> packI32(const SmallVector<Value> &inValues, Type srcTy, | |||
| } | |||
| return outValues; | |||
| } | |||
|
|
|||
| int getNumElmenetPerThreads(Type type, const LLVMTypeConverter *typeConverter) { | |||
There was a problem hiding this comment.
This function is only correct for inline asm, right? If so, can we change the name to indicate that? Also can we add a comment indicating why we do 32/size? (It's because inline asm implicitly packs elements in this way, I believe.)
There was a problem hiding this comment.
no I think the function should work for any op.
Also can we add a comment indicating why we do 32/size?
sure
| // need to reorder them so we iterate over the operands' elements in the | ||
| // same logical order. | ||
| for (unsigned i = 0; i < unpackedOperands.size(); ++i) { | ||
| unpackedOperands[i] = reorderValues( |
There was a problem hiding this comment.
Hm, where did this call to reorderValues go?
There was a problem hiding this comment.
it was not correct, removed it
address some comments from #3521
Multiple fixes to allow pipelining to happen when generating a matmul with fp8e4b15 inputs. Also clean useless code in the frontend.