Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle > 64 registers + predicate registers for Arm64 #98258

Closed
wants to merge 212 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
212 commits
Select commit Hold shift + click to select a range
dcb4b16
Introduce regMaskGpr, regMaskFloat, regMaskPredicate
kunalspathak Feb 7, 2024
87b0f97
Add IsGpr() and IsFloat()
kunalspathak Feb 7, 2024
ba62855
Renamed to regMaskOnlyOne and regMaskAny
kunalspathak Feb 7, 2024
375bf8e
convert from #define to typedef for better intellisense support
kunalspathak Feb 7, 2024
da71584
Make IsGprRegMask() and IsFloatRegMask()
kunalspathak Feb 7, 2024
0ba172e
Update codegen.h and related code
kunalspathak Feb 7, 2024
8639bb7
Update codegenarm.cpp and related code
kunalspathak Feb 7, 2024
3c5bdaf
Update codegenarm64.cpp and related code
kunalspathak Feb 7, 2024
bad9e8b
Update codegenarmarch.cpp and related code
kunalspathak Feb 7, 2024
8384ff2
Partial update codegencommon.cpp
kunalspathak Feb 8, 2024
c78dc4b
Update all gcRef and gcByRef to regMaskGpr
kunalspathak Feb 9, 2024
e05cece
Update codegencommon.cpp and related code
kunalspathak Feb 9, 2024
82ed9aa
Update codegeninterface.h and related code
kunalspathak Feb 9, 2024
91462c7
Update codegenxarch.cpp and related code
kunalspathak Feb 9, 2024
b9d1ed5
Update compiler.cpp and related code
kunalspathak Feb 9, 2024
83f0f01
Update compiler.h and related code
kunalspathak Feb 9, 2024
c3797e0
Update compiler.hpp and related code
kunalspathak Feb 9, 2024
147891c
Update emit.cpp and related code
kunalspathak Feb 9, 2024
2238e41
Update emit.h and related code
kunalspathak Feb 9, 2024
e5e71d1
Update emitxarch.cpp and related code
kunalspathak Feb 9, 2024
7e15435
Update gentree.cpp and related code
kunalspathak Feb 9, 2024
6bbc2d2
Update gentree.h and related code
kunalspathak Feb 9, 2024
3e3d51a
Update lclvars.cpp and related code
kunalspathak Feb 9, 2024
41959f2
Update morph.cpp and related code
kunalspathak Feb 9, 2024
4b72781
Update regalloc.cpp and related code
kunalspathak Feb 9, 2024
7214d7b
Update registerargconvention.h and related code
kunalspathak Feb 9, 2024
c7e913e
Update regset.cpp and related code
kunalspathak Feb 9, 2024
d417ad5
Update regset.h and related code
kunalspathak Feb 9, 2024
dc54b20
Update target.h and related code
kunalspathak Feb 9, 2024
5b8a1e6
Update targetamd64.h and related code
kunalspathak Feb 9, 2024
c79e9fe
Update targetarm.cpp and related code
kunalspathak Feb 9, 2024
9171cd8
Update targetarm.h and related code
kunalspathak Feb 9, 2024
0aeefe1
Update targetarm64.cpp and related code
kunalspathak Feb 9, 2024
9ccaabb
Update targetarm64.h and related code
kunalspathak Feb 9, 2024
806fc03
Update targetx86.cpp and related code
kunalspathak Feb 9, 2024
757d8f2
Update targetx86.h and related code
kunalspathak Feb 9, 2024
e6b3304
Update unwindarmarch.cpp and related code
kunalspathak Feb 9, 2024
48e2e32
Update utils.cpp and related code
kunalspathak Feb 9, 2024
93f87fa
Add singleRegMask and some notes in target.h
kunalspathak Feb 10, 2024
f2c8219
Update lsra.cpp and related code
kunalspathak Feb 10, 2024
8e97320
Update lsra.h and related code
kunalspathak Feb 10, 2024
1b24a61
fix compilation error for singleRegMask
kunalspathak Feb 10, 2024
1842bc2
Update lsraarm.cpp and related code
kunalspathak Feb 10, 2024
d3d81fe
Update lsraarm64.cpp and related code
kunalspathak Feb 10, 2024
43a19d6
Update lsraarmarch.cpp and related code
kunalspathak Feb 10, 2024
cdad314
Update lsrabuild.cpp and related code
kunalspathak Feb 10, 2024
7937068
Update lsraxarch.cpp and related code
kunalspathak Feb 10, 2024
82000a5
Convert the return type of genRegMask() from `regMaskAny` to `singleR…
kunalspathak Feb 10, 2024
515383b
Rename regMaskAny to regMaskMixed
kunalspathak Feb 10, 2024
3fb09bb
Update design notes of target.h
kunalspathak Feb 10, 2024
cdb42bc
fix the typo
kunalspathak Feb 10, 2024
e2cc19b
Update the assert in IsOnlyOneRegMask()
kunalspathak Feb 10, 2024
656629e
Fix assert in buildUpperVectorSaveRefPositions()
kunalspathak Feb 10, 2024
ec471e5
Revert to buildUpperVectorSaveRefPositions() to regMaskFloat
kunalspathak Feb 12, 2024
ac8b5ea
Change BuildDefs() to regMaskMixed
kunalspathak Feb 12, 2024
8e17746
jit format
kunalspathak Feb 12, 2024
0659741
Convert m_RegistersWithConstants to regMaskOnlyOne
kunalspathak Feb 12, 2024
15135f8
temp: Convert m_AvailableRegs to regMaskOnlyOne
kunalspathak Feb 13, 2024
4c6cfda
Add a TODO to convert m_Regs to separate variables
kunalspathak Feb 13, 2024
e08561b
Handle various `regsModified` and `RBM_` to return AllRegsMask
kunalspathak Feb 16, 2024
b1cad44
Update genPoisonFrame()
kunalspathak Feb 16, 2024
65dbc13
Merge remote-tracking branch 'origin/main' into predicate-registers
kunalspathak Feb 16, 2024
dabf8f5
Fix some of the reserved registers and related code paths
kunalspathak Feb 29, 2024
b3febfa
Add AllRegsMask for *VarRegs() method
kunalspathak Mar 1, 2024
16cdcf9
Remove regMaskMixed from emitxarch.cpp
kunalspathak Mar 1, 2024
2849c8d
Remove regMaskMixed from *gcinfo.*
kunalspathak Mar 1, 2024
eeb173a
Remove regMaskMixed from TempReg()
kunalspathak Mar 1, 2024
500b78a
Handled unwind.cpp
kunalspathak Mar 1, 2024
ae53e67
Remove regMaskMixed from misc places
kunalspathak Mar 1, 2024
03eab41
fix unwind build error
kunalspathak Mar 2, 2024
729debf
fix the handling of RemoveMaskVars()
kunalspathak Mar 4, 2024
951e9c6
Fix makeRegAvailable and setRegInUse for registerType/reg inconsistency
kunalspathak Mar 5, 2024
c66b485
fix some printing the dumps
kunalspathak Mar 5, 2024
2017d52
Add support for BuildKills/BuildDefs/BuildCall
kunalspathak Mar 6, 2024
90883b8
gentree: gtGetRegMask() and updating Npt
kunalspathak Mar 6, 2024
931001d
Remove regMaskMixed from lsraarmarch.cpp
kunalspathak Mar 8, 2024
bda4c07
Remove regMaskMixed lsra: resolveEdge, tempRegForResolution
kunalspathak Mar 8, 2024
2cddfdb
Remove regMaskMixed - lsra:getConstrainedRegMask()
kunalspathak Mar 8, 2024
a353db4
Remove regMaskMixed - lsra:freeRegister
kunalspathak Mar 10, 2024
a767406
Remove regMaskMixed - lsra:fixedRegs, registersToDump, etc.
kunalspathak Mar 11, 2024
fba4c09
Fix some typo issues
kunalspathak Mar 11, 2024
7401f00
Remove regMaskMixed: getMatchingConstants
kunalspathak Mar 11, 2024
0d6a0bf
Remove regMaskMixed: placedArgRegs
kunalspathak Mar 11, 2024
91ae489
Remove regMaskMixed from lsra.h
kunalspathak Mar 11, 2024
3d3938e
Remove regMaskMixed : lsra.cpp mostly
kunalspathak Mar 11, 2024
f2b172d
Remove regMaskMixed: lsra.cpp/resolveEdge/targetRegsToDo
kunalspathak Mar 11, 2024
0f0c60e
Remove regMaskMixed: lsra.cpp/resolveEdge/targetRegsReady
kunalspathak Mar 11, 2024
2fbef79
Remove regMaskMixed: lsra.cpp/resolveEdge/targetRegsFromStack
kunalspathak Mar 11, 2024
dbb9be3
Complete the remaining ones
kunalspathak Mar 11, 2024
1dd8e1b
Merge remote-tracking branch 'origin/main' into predicate-registers
kunalspathak Mar 11, 2024
1388622
merge errors
kunalspathak Mar 11, 2024
9c4942d
jit format
kunalspathak Mar 11, 2024
15c482f
fix some build errors and failures
kunalspathak Mar 11, 2024
37b6cce
Remove the assert(false)
kunalspathak Mar 12, 2024
e89a3ee
Pass dstCount for BuildDefWithKills()
kunalspathak Mar 12, 2024
a2c99a0
Fix dspRegMask for compilation errors:
kunalspathak Mar 12, 2024
3c80871
fix some more asserts
kunalspathak Mar 12, 2024
9e597bc
fix arm issue
kunalspathak Mar 12, 2024
badf863
jit format
kunalspathak Mar 12, 2024
27e6bdf
fix the asmdiff for windows/x64
kunalspathak Mar 12, 2024
72ce973
Merge remote-tracking branch 'origin/main' into merge-predicate-regis…
kunalspathak Mar 12, 2024
e7ee960
fix the JITStressRegs AV
kunalspathak Mar 13, 2024
85ed5a6
Introduce rbmAllFloats in emit*
kunalspathak Mar 13, 2024
9199913
fix the masks creation
kunalspathak Mar 13, 2024
3db212d
Merge remote-tracking branch 'kp/predicate-registers' into predicate-…
kunalspathak Mar 13, 2024
ab4ec81
Condition free code: reduce TP from 5% -> 3%
kunalspathak Mar 16, 2024
b1bbbbf
Some rename methods of AllRegsMask
kunalspathak Mar 16, 2024
5ca0332
fix some issues
kunalspathak Mar 18, 2024
502acd8
Merge remote-tracking branch 'origin' into predicate-registers
kunalspathak Mar 18, 2024
f283c30
fix merge conflict issues
kunalspathak Mar 18, 2024
e936cb7
Refactoring
kunalspathak Mar 19, 2024
150dd94
jit format
kunalspathak Mar 19, 2024
679cca8
Make the IsGprRegMask() and IsFloatRegMask() checks more robust
kunalspathak Mar 19, 2024
6e48e2a
Mark bunch of methods as FORCEINLINE, added TODO for others
kunalspathak Mar 19, 2024
e6e2269
Revert "Mark bunch of methods as FORCEINLINE, added TODO for others"
kunalspathak Mar 20, 2024
6af17c4
Change RegSet to use AllRegMask branch-free
kunalspathak Mar 20, 2024
ce78527
Misc change around rsRegsModified
kunalspathak Mar 20, 2024
314c464
Make some methods out of #ifdef DEBUG
kunalspathak Mar 20, 2024
973e14e
Pass AllRegsMask& by reference
kunalspathak Mar 20, 2024
4551999
Fix the byref problem
kunalspathak Mar 21, 2024
522fbd8
Mark AllRegsMask operators as FORCEINLINE
kunalspathak Mar 21, 2024
020b709
Mark AllRegsMask methods as FORCEINLINE
kunalspathak Mar 21, 2024
31a940e
Prototype of union using _float_gpr
kunalspathak Mar 24, 2024
8649987
hide predicate logic under #ifdef to see TP impact. It is just 0.5% r…
kunalspathak Mar 25, 2024
25bdad9
Remove GetMaskForRegNum()
kunalspathak Mar 26, 2024
e5d5e3b
fix the creation of certain masks
kunalspathak Mar 26, 2024
951b260
Introduce HAS_MORE_THAN_64_REGISTERS
kunalspathak Mar 26, 2024
33063eb
Introduce _hasPredicateRegister
kunalspathak Mar 28, 2024
ed29d62
fix some bugs
kunalspathak Mar 30, 2024
896fb35
Merge remote-tracking branch 'origin/main' into predicate/more-than-64
kunalspathak Mar 30, 2024
8ee87f4
fix some merge conflict
kunalspathak Mar 30, 2024
f1ae83c
Eliminate HAS_PREDICATE_REGS with FEATURE_MASKED_HW_INTRINSICS/HAS_MO…
kunalspathak Mar 31, 2024
6c2eb77
jit format
kunalspathak Mar 31, 2024
af618db
Merge remote-tracking branch 'origin/main' into predicate/more-than-64
kunalspathak Apr 1, 2024
f94e6cf
Arm64: Have predicate registers from 64~79
kunalspathak Apr 1, 2024
d560200
Add FEATURE_MASKED_HW_INSTRINSICS at few more places
kunalspathak Apr 1, 2024
5ff6d61
Have own version of updateDeadCandidatesAtBlockStart for AllRegsMask
kunalspathak Apr 1, 2024
8d6a2b6
Fix resetting availableMaskRegs
kunalspathak Apr 1, 2024
57a0746
Remove some unneeded methods from AllRegsMask
kunalspathak Apr 1, 2024
1e960b4
Arm64: Update predicate callee-saved and callee-trash registers
kunalspathak Apr 1, 2024
adc51ac
update save/restore of predicate registers on arm64
kunalspathak Apr 1, 2024
06e31ec
Remove _hasPredicateRegister
kunalspathak Apr 1, 2024
e4700f8
fix gprRegs()
kunalspathak Apr 2, 2024
6763c70
Restruture AllRegsMask to remove build errors for linux
kunalspathak Apr 2, 2024
5a69528
Replace _float_gpr and _allRegisters with _combinedRegisters
kunalspathak Apr 2, 2024
704fcb1
fix assert in arm64
kunalspathak Apr 2, 2024
dda6d8a
fix some #define and remove Create_AllRegsMask()
kunalspathak Apr 2, 2024
8adc9f7
fix arm build break
kunalspathak Apr 2, 2024
4461bdd
Arm64: Increase REGNUM_BITS 6->7
kunalspathak Apr 2, 2024
5d4f507
fix sve unit test
kunalspathak Apr 2, 2024
fc1ff92
fix osx/arm64 replay failures
kunalspathak Apr 2, 2024
f27b326
fix dumpRegMask()
kunalspathak Apr 3, 2024
e9f0a46
Introduce MORE_THAN_64_REG_ARG
kunalspathak Apr 3, 2024
f6f1e03
Update genRegNumFromMask() to take MORE_THAN_64_REG_ARG
kunalspathak Apr 3, 2024
e808d0f
Update genFirstRegNumFromMask() to take MORE_THAN_64_REG_ARG
kunalspathak Apr 3, 2024
cdabdcf
Handle TYP_MASK in getTempRegForResolution()
kunalspathak Apr 3, 2024
47c683f
Add assert in genUpdateRegLife()
kunalspathak Apr 3, 2024
611ed21
Update genFirstRegNumFromMaskAndToggle() to take MORE_THAN_64_REG_ARG
kunalspathak Apr 3, 2024
b29b417
fix freeRegisterMask() typo
kunalspathak Apr 3, 2024
25b9893
Display predicate registers in Allocation table
kunalspathak Apr 3, 2024
952af0d
make gprRegs() use RBM_ALLINT
kunalspathak Apr 3, 2024
ce370e3
Fix JITDump for AllMask
kunalspathak Apr 3, 2024
6f96b57
make floatRegs() and predicateRegs() use RBM_ALLFLOAT/RBM_ALLMASK
kunalspathak Apr 3, 2024
ff56a4f
fix the dumping of gpr/float registers
kunalspathak Apr 4, 2024
bff97e5
Introduce RBM_ALLGPR
kunalspathak Apr 4, 2024
49194e1
move getRegType from lsra.h -> target.h
kunalspathak Apr 4, 2024
9041b49
Rewrite AddRegNum/RemoveRegNum/IsRegNum for arm
kunalspathak Apr 4, 2024
ccba323
jit format 17.0.6
kunalspathak Apr 4, 2024
1fefa86
Merge remote-tracking branch 'origin/main' into temp
kunalspathak Apr 4, 2024
82a3518
jit format 17.0.6
kunalspathak Apr 4, 2024
3407277
Merge remote-tracking branch 'origin/main' into predicate/more-than-64
kunalspathak Apr 4, 2024
e43bd80
jit format
kunalspathak Apr 4, 2024
4f546c6
some fixes + formatting
kunalspathak Apr 5, 2024
7657166
Merge remote-tracking branch 'origin/main' into predicate/more-than-64
kunalspathak Apr 5, 2024
a1f157b
Fix changes from swift
kunalspathak Apr 5, 2024
40bb3b1
Fix the argument for AllRegsMask()
kunalspathak Apr 5, 2024
506dc9f
Remove TODO #1
kunalspathak Apr 7, 2024
847701c
Remove TODO #2
kunalspathak Apr 9, 2024
1289175
Refactor freeRegsiterMask()
kunalspathak Apr 9, 2024
5ba1773
Return `RegBit64` for gtGetContainedRegMask(), gtGetRegMask()
kunalspathak Apr 9, 2024
4f4c317
Remove another TODO
kunalspathak Apr 9, 2024
dc6ff1f
Remaining gtGetContainedRegMask() fix
kunalspathak Apr 9, 2024
80faa40
Move registerType to Referencable
kunalspathak Apr 9, 2024
204d17b
Mark some of the AllRegsMask as const
kunalspathak Apr 9, 2024
5d16126
Replace AllRegsMask() with AllRegsMask_NONE
kunalspathak Apr 9, 2024
99de237
Added CONSTREF and REF to pass AllRegsMask by (const) reference
kunalspathak Apr 9, 2024
13819f6
Merge remote-tracking branch 'origin/main' into predicate/more-than-64
kunalspathak Apr 9, 2024
baf4e36
fix the merge conflicts from parameter homing
kunalspathak Apr 9, 2024
78614fa
jit format
kunalspathak Apr 9, 2024
ad513dc
Move from target definition to compiler object
kunalspathak Apr 10, 2024
c38cfdb
Use AllRegsMask_* from compiler object
kunalspathak Apr 10, 2024
3b1eeed
WIP: Make methods return CONSTREF
kunalspathak Apr 10, 2024
d8ba174
Introduce compInitAllRegsMask()
kunalspathak Apr 10, 2024
0ccb59a
Remove rsAllCalleeSavedMask from RegSet
kunalspathak Apr 10, 2024
e27691c
Revert "Remove rsAllCalleeSavedMask from RegSet"
kunalspathak Apr 10, 2024
dc96a99
Move the initialization of rbmAllFloat inside compInitAllRegsMask()
kunalspathak Apr 10, 2024
cba4510
Reapply "Remove rsAllCalleeSavedMask from RegSet"
kunalspathak Apr 10, 2024
fc6b76f
Move codeGen->CopyRegisterInfo() in compInitAllRegsMask()
kunalspathak Apr 10, 2024
3df5a5d
Copy calleeTrashRegs info as well
kunalspathak Apr 10, 2024
c591106
Make emitGetGCRegsKilledByNoGCCall() return CONSTREF_
kunalspathak Apr 10, 2024
72f27e8
fix some more CONSTREF
kunalspathak Apr 10, 2024
e1823d4
Consolidate updateDeadCandidates and inActivateRegisters
kunalspathak Apr 10, 2024
41d49af
Replace AllRegsMask_NONE with .Clear()
kunalspathak Apr 10, 2024
8e448c5
Add AllRegsMask in clrjit.natvis:
kunalspathak Apr 10, 2024
e5eb8e4
Remove the commented out #define
kunalspathak Apr 10, 2024
7e522f1
Make emitGetGCRegsKilledByNoGCCall() return RegbitSet64 instead of Al…
kunalspathak Apr 10, 2024
3eb1237
emitGetGCRegsSaveOrModified() returns RegBitSet64
kunalspathak Apr 10, 2024
335b2b5
add back some RBM_* in target**
kunalspathak Apr 10, 2024
c60f3b1
misc. cleanup
kunalspathak Apr 10, 2024
7042510
jit format
kunalspathak Apr 10, 2024
22077eb
fix linux-x86 build
kunalspathak Apr 10, 2024
893f6be
fix linux-x64 Native_GCC build error
kunalspathak Apr 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
jit format
  • Loading branch information
kunalspathak committed Mar 11, 2024
commit 9c4942df40832bc8115e18ccda1512d0c71340d3
5 changes: 3 additions & 2 deletions src/coreclr/jit/codegen.h
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@ class CodeGen final : public CodeGenInterface
// same.
struct FuncletFrameInfoDsc
{
regMaskGpr fiSaveGprRegs; // Set of GPR registers saved in the funclet prolog (includes LR)
regMaskGpr fiSaveGprRegs; // Set of GPR registers saved in the funclet prolog (includes LR)
regMaskFloat fiSaveFloatRegs; // Set of Float registers saved in the funclet prolog (includes LR)
unsigned fiFunctionCallerSPtoFPdelta; // Delta between caller SP and the frame pointer
unsigned fiSpDelta; // Stack pointer delta
Expand All @@ -404,7 +404,8 @@ class CodeGen final : public CodeGenInterface
{
regMaskGpr fiSaveGprRegs; // Set of callee-saved GPR registers saved in the funclet prolog (includes LR)
regMaskFloat fiSaveFloatRegs; // Set of callee-saved float registers saved in the funclet prolog (includes LR)
regMaskFloat fiSavePredicateRegs; // Set of callee-saved predicate registers saved in the funclet prolog (includes LR)
regMaskFloat fiSavePredicateRegs; // Set of callee-saved predicate registers saved in the funclet prolog
// (includes LR)
int fiFunction_CallerSP_to_FP_delta; // Delta between caller SP and the frame pointer in the parent function
// (negative)
int fiSP_to_FPLR_save_delta; // FP/LR register save offset from SP (positive)
Expand Down
18 changes: 9 additions & 9 deletions src/coreclr/jit/codegenarm.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2472,11 +2472,11 @@ void CodeGen::genCaptureFuncletPrologEpilogInfo()
unsigned preSpillRegArgSize = genCountBits(regSet.rsMaskPreSpillRegs(true)) * REGSIZE_BYTES;
genFuncletInfo.fiFunctionCallerSPtoFPdelta = preSpillRegArgSize + 2 * REGSIZE_BYTES;

regMaskGpr rsGprMaskSaveRegs = regSet.rsGprMaskCalleeSaved;
regMaskFloat rsFloatMaskSaveRegs = regSet.rsFloatMaskCalleeSaved;
regMaskGpr rsGprMaskSaveRegs = regSet.rsGprMaskCalleeSaved;
regMaskFloat rsFloatMaskSaveRegs = regSet.rsFloatMaskCalleeSaved;
unsigned saveRegsCount = genCountBits(rsGprMaskSaveRegs) + genCountBits(rsFloatMaskSaveRegs);
unsigned saveRegsSize = saveRegsCount * REGSIZE_BYTES; // bytes of regs we're saving
unsigned saveSizeWithPSP = saveRegsSize + REGSIZE_BYTES /* PSP sym */;
unsigned saveRegsSize = saveRegsCount * REGSIZE_BYTES; // bytes of regs we're saving
unsigned saveSizeWithPSP = saveRegsSize + REGSIZE_BYTES /* PSP sym */;
if (compiler->lvaMonAcquired != BAD_VAR_NUM)
{
saveSizeWithPSP += TARGET_POINTER_SIZE;
Expand All @@ -2494,8 +2494,8 @@ void CodeGen::genCaptureFuncletPrologEpilogInfo()

/* Now save it for future use */

genFuncletInfo.fiSaveGprRegs = rsGprMaskSaveRegs;
genFuncletInfo.fiSaveFloatRegs = rsFloatMaskSaveRegs;
genFuncletInfo.fiSaveGprRegs = rsGprMaskSaveRegs;
genFuncletInfo.fiSaveFloatRegs = rsFloatMaskSaveRegs;
genFuncletInfo.fiSpDelta = spDelta;
genFuncletInfo.fiPSP_slot_SP_offset = PSP_slot_SP_offset;
genFuncletInfo.fiPSP_slot_CallerSP_offset = PSP_slot_CallerSP_offset;
Expand Down Expand Up @@ -2628,9 +2628,9 @@ void CodeGen::genZeroInitFrameUsingBlockInit(int untrLclHi, int untrLclLo, regNu
//
// <optional> str rZero1,[rAddr] // When cnt is odd

regNumber rAddr;
regNumber rCnt = REG_NA; // Invalid
regMaskGpr regMask;
regNumber rAddr;
regNumber rCnt = REG_NA; // Invalid
regMaskGpr regMask;

regMaskGpr availMask = regSet.rsGetModifiedGprRegsMask() | RBM_INT_CALLEE_TRASH; // Set of available registers
availMask &= ~intRegState.rsCalleeRegArgMaskLiveIn; // Remove all of the incoming argument registers as they are
Expand Down
24 changes: 12 additions & 12 deletions src/coreclr/jit/codegenarm64.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ void CodeGen::genPopCalleeSavedRegistersAndFreeLclFrame(bool jmpEpilog)
{
assert(compiler->compGeneratingEpilog);

regMaskGpr rsRestoreGprRegs = regSet.rsGetModifiedGprRegsMask() & RBM_INT_CALLEE_SAVED;
regMaskGpr rsRestoreGprRegs = regSet.rsGetModifiedGprRegsMask() & RBM_INT_CALLEE_SAVED;
regMaskFloat rsRestoreFloatRegs = regSet.rsGetModifiedFloatRegsMask() & RBM_FLT_CALLEE_SAVED;

if (isFramePointerUsed())
Expand Down Expand Up @@ -1022,12 +1022,12 @@ void CodeGen::genRestoreCalleeSavedRegisterGroup(regMaskOnlyOne regsMask, int sp
// None.

void CodeGen::genRestoreCalleeSavedRegistersHelp(AllRegsMask regsToRestoreMask,
int lowestCalleeSavedOffset,
int spDelta)
int lowestCalleeSavedOffset,
int spDelta)
{
assert(spDelta >= 0);

regMaskGpr maskRestoreRegsInt = regsToRestoreMask.gprRegs;
regMaskGpr maskRestoreRegsInt = regsToRestoreMask.gprRegs;
regMaskFloat maskRestoreRegsFloat = regsToRestoreMask.floatRegs;

assert(compiler->IsGprRegMask(maskRestoreRegsInt));
Expand Down Expand Up @@ -1743,8 +1743,8 @@ void CodeGen::genCaptureFuncletPrologEpilogInfo()

genFuncletInfo.fiFunction_CallerSP_to_FP_delta = genCallerSPtoFPdelta() - osrPad;

regMaskGpr rsMaskSaveGprRegs = regSet.rsGprMaskCalleeSaved;
regMaskFloat rsMaskSaveFloatRegs = regSet.rsFloatMaskCalleeSaved;
regMaskGpr rsMaskSaveGprRegs = regSet.rsGprMaskCalleeSaved;
regMaskFloat rsMaskSaveFloatRegs = regSet.rsFloatMaskCalleeSaved;
regMaskPredicate rsMaskSavePredicateRegs = RBM_NONE;
#ifdef HAS_PREDICATE_REGS
rsMaskSavePredicateRegs = regSet.rsPredicateMaskCalleeSaved;
Expand Down Expand Up @@ -1873,9 +1873,9 @@ void CodeGen::genCaptureFuncletPrologEpilogInfo()

/* Now save it for future use */

genFuncletInfo.fiSaveGprRegs = rsMaskSaveGprRegs;
genFuncletInfo.fiSaveFloatRegs = rsMaskSaveFloatRegs;
genFuncletInfo.fiSavePredicateRegs = rsMaskSavePredicateRegs;
genFuncletInfo.fiSaveGprRegs = rsMaskSaveGprRegs;
genFuncletInfo.fiSaveFloatRegs = rsMaskSaveFloatRegs;
genFuncletInfo.fiSavePredicateRegs = rsMaskSavePredicateRegs;
genFuncletInfo.fiSP_to_FPLR_save_delta = SP_to_FPLR_save_delta;
genFuncletInfo.fiSP_to_PSP_slot_delta = SP_to_PSP_slot_delta;
genFuncletInfo.fiSP_to_CalleeSave_delta = SP_to_PSP_slot_delta + PSPSize;
Expand All @@ -1887,8 +1887,8 @@ void CodeGen::genCaptureFuncletPrologEpilogInfo()
printf("\n");
printf("Funclet prolog / epilog info\n");
printf(" Save regs: ");
dspRegMask(
AllRegsMask(genFuncletInfo.fiSaveGprRegs, genFuncletInfo.fiSaveFloatRegs, genFuncletInfo.fiSavePredicateRegs));
dspRegMask(AllRegsMask(genFuncletInfo.fiSaveGprRegs, genFuncletInfo.fiSaveFloatRegs,
genFuncletInfo.fiSavePredicateRegs));
printf("\n");
if (compiler->opts.IsOSR())
{
Expand Down Expand Up @@ -5132,7 +5132,7 @@ void CodeGen::genEmitHelperCall(unsigned helper, int argSize, emitAttr retSize,
callTargetReg = REG_DEFAULT_HELPER_CALL_TARGET;
}

regMaskGpr callTargetMask = genRegMask(callTargetReg);
regMaskGpr callTargetMask = genRegMask(callTargetReg);
AllRegsMask callKillSet = compiler->compHelperCallKillSet((CorInfoHelpFunc)helper);

// assert that all registers in callTargetMask are in the callKillSet
Expand Down
12 changes: 6 additions & 6 deletions src/coreclr/jit/codegenarmarch.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3526,7 +3526,7 @@ void CodeGen::genCallInstruction(GenTreeCall* call)

if (call->IsFastTailCall())
{
regMaskGpr trashedGprByEpilog = RBM_INT_CALLEE_SAVED;
regMaskGpr trashedGprByEpilog = RBM_INT_CALLEE_SAVED;
regMaskFloat trashedFloatByEpilog = RBM_FLT_CALLEE_SAVED;
#ifdef HAS_PREDICATE_REGS
regMaskPredicate trashedPredicateByEpilog = RBM_MSK_CALLEE_SAVED;
Expand Down Expand Up @@ -3795,7 +3795,7 @@ void CodeGen::genJmpMethod(GenTree* jmp)
// are not frequent.
for (varNum = 0; varNum < compiler->info.compArgsCount; varNum++)
{
varDsc = compiler->lvaGetDesc(varNum);
varDsc = compiler->lvaGetDesc(varNum);
regNumber varReg = varDsc->GetRegNum();

if (varDsc->lvPromoted)
Expand Down Expand Up @@ -4902,7 +4902,7 @@ void CodeGen::genPushCalleeSavedRegisters()
intRegState.rsCalleeRegArgMaskLiveIn);
#endif

regMaskGpr rsPushGprRegs = regSet.rsGetModifiedGprRegsMask() & RBM_INT_CALLEE_SAVED;
regMaskGpr rsPushGprRegs = regSet.rsGetModifiedGprRegsMask() & RBM_INT_CALLEE_SAVED;
regMaskFloat rsPushFloatRegs = regSet.rsGetModifiedFloatRegsMask() & RBM_FLT_CALLEE_SAVED;

#if ETW_EBP_FRAMED
Expand Down Expand Up @@ -4937,7 +4937,7 @@ void CodeGen::genPushCalleeSavedRegisters()
//
rsPushGprRegs |= RBM_LR; // We must save the return address (in the LR register)

regSet.rsGprMaskCalleeSaved = rsPushGprRegs;
regSet.rsGprMaskCalleeSaved = rsPushGprRegs;
regSet.rsFloatMaskCalleeSaved = rsPushFloatRegs;

#ifdef DEBUG
Expand Down Expand Up @@ -5550,8 +5550,8 @@ void CodeGen::genFnEpilog(BasicBlock* block)
}

if (jmpEpilog ||
genStackAllocRegisterMask(compiler->compLclFrameSize, regSet.rsGetModifiedFloatRegsMask() & RBM_FLT_CALLEE_SAVED) ==
RBM_NONE)
genStackAllocRegisterMask(compiler->compLclFrameSize,
regSet.rsGetModifiedFloatRegsMask() & RBM_FLT_CALLEE_SAVED) == RBM_NONE)
{
genFreeLclFrame(compiler->compLclFrameSize, &unwindStarted);
}
Expand Down
57 changes: 31 additions & 26 deletions src/coreclr/jit/codegencommon.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5328,7 +5328,7 @@ void CodeGen::genFinalizeFrame()
if (compiler->compLclFrameSize >= compiler->eeGetPageSize())
{
regSet.rsSetGprRegsModified(RBM_STACK_PROBE_HELPER_ARG | RBM_STACK_PROBE_HELPER_CALL_TARGET |
RBM_STACK_PROBE_HELPER_TRASH);
RBM_STACK_PROBE_HELPER_TRASH);
}

// If there are any reserved registers, add them to the modified set.
Expand All @@ -5344,9 +5344,10 @@ void CodeGen::genFinalizeFrame()
printf("Modified regs: ");
dspRegMask(AllRegsMask(regSet.rsGetModifiedGprRegsMask(), regSet.rsGetModifiedFloatRegsMask()
#ifdef HAS_PREDICATE_REGS
,regSet.rsGetModifiedPredicateRegsMask()
,
regSet.rsGetModifiedPredicateRegsMask()
#endif
));
));
printf("\n");
}
#endif // DEBUG
Expand Down Expand Up @@ -5399,7 +5400,7 @@ void CodeGen::genFinalizeFrame()
#endif

#ifdef TARGET_ARM
// TODO-ARM64-Bug?: enable some variant of this for FP on ARM64?
// TODO-ARM64-Bug?: enable some variant of this for FP on ARM64?
#endif
regMaskFloat maskPushRegsInt = regSet.rsGetModifiedGprRegsMask() & RBM_INT_CALLEE_SAVED;
regMaskGpr maskPushRegsFloat = regSet.rsGetModifiedFloatRegsMask() & RBM_FLT_CALLEE_SAVED;
Expand Down Expand Up @@ -5495,7 +5496,7 @@ void CodeGen::genFinalizeFrame()
if (verbose)
{
printf("Callee-saved registers pushed: %d ", compiler->compCalleeRegsPushed);
dspRegMask(maskPushRegsInt, maskPushRegsFloat); //TODO: Should this also have maskPushRegsPredicate
dspRegMask(maskPushRegsInt, maskPushRegsFloat); // TODO: Should this also have maskPushRegsPredicate
printf("\n");
}
#endif // DEBUG
Expand Down Expand Up @@ -5834,9 +5835,9 @@ void CodeGen::genFnProlog()
// Track if initReg holds non-zero value. Start conservative and assume it has non-zero value.
// If initReg is ever set to zero, this variable is set to true and zero initializing initReg
// will be skipped.
bool initRegZeroed = false;
bool initRegZeroed = false;
regMaskGpr excludeMask = intRegState.rsCalleeRegArgMaskLiveIn;
regMaskGpr tempMask;
regMaskGpr tempMask;

// We should not use the special PINVOKE registers as the initReg
// since they are trashed by the jithelper call to setup the PINVOKE frame
Expand Down Expand Up @@ -8188,16 +8189,16 @@ void CodeGen::genRegCopy(GenTree* treeNode)
unsigned regCount = op1->GetMultiRegCount(compiler);
assert(regCount <= MAX_MULTIREG_COUNT);

// First set the source registers as busy if they haven't been spilled.
// (Note that this is just for verification that we don't have circular dependencies.)
// First set the source registers as busy if they haven't been spilled.
// (Note that this is just for verification that we don't have circular dependencies.)
#ifdef DEBUG
AllRegsMask busyRegs;
for (unsigned i = 0; i < regCount; ++i)
{
if ((op1->GetRegSpillFlagByIdx(i) & GTF_SPILLED) == 0)
{
regNumber reg = op1->GetRegByIndex(i);
//regMaskOnlyOne regMask = genRegMask();
regNumber reg = op1->GetRegByIndex(i);
// regMaskOnlyOne regMask = genRegMask();
if (genIsValidIntReg(reg))
{
busyRegs.gprRegs |= genRegMask(reg);
Expand Down Expand Up @@ -8225,13 +8226,17 @@ void CodeGen::genRegCopy(GenTree* treeNode)

#ifdef DEBUG

#define DO_VALIDATION(regType) \
if (targetReg != sourceReg) \
{ \
singleRegMask targetRegMask = genRegMask(targetReg); \
assert((busyRegs.regType & targetRegMask) == 0); \
busyRegs.regType &= ~genRegMask(sourceReg); \
} \
#define DO_VALIDATION(regType) \
\
if(targetReg != sourceReg) \
\
{ \
singleRegMask targetRegMask = genRegMask(targetReg); \
assert((busyRegs.regType & targetRegMask) == 0); \
busyRegs.regType &= ~genRegMask(sourceReg); \
\
} \
\
busyRegs.regType |= genRegMask(targetReg);

if (genIsValidIntReg(targetReg))
Expand All @@ -8248,14 +8253,14 @@ busyRegs.regType |= genRegMask(targetReg);
DO_VALIDATION(predicateRegs)
}
#endif // HAS_PREDICATE_REGS
//if (targetReg != sourceReg)
//{
// singleRegMask targetRegMask = genRegMask(targetReg);
// assert((busyRegs & targetRegMask) == 0);
// // Clear sourceReg from the busyRegs, and add targetReg.
// busyRegs &= ~genRegMask(sourceReg);
//}
//busyRegs |= genRegMask(targetReg);
// if (targetReg != sourceReg)
//{
// singleRegMask targetRegMask = genRegMask(targetReg);
// assert((busyRegs & targetRegMask) == 0);
// // Clear sourceReg from the busyRegs, and add targetReg.
// busyRegs &= ~genRegMask(sourceReg);
//}
// busyRegs |= genRegMask(targetReg);
#endif // DEBUG
}
return;
Expand Down
4 changes: 2 additions & 2 deletions src/coreclr/jit/codegeninterface.h
Original file line number Diff line number Diff line change
Expand Up @@ -158,8 +158,8 @@ class CodeGenInterface
bool genTempLiveChg;
#endif

VARSET_TP genLastLiveSet; // A one element map (genLastLiveSet-> genLastLiveMask)
regMaskTP genLastLiveMask; // these two are used in genLiveMask
VARSET_TP genLastLiveSet; // A one element map (genLastLiveSet-> genLastLiveMask)
regMaskTP genLastLiveMask; // these two are used in genLiveMask

regMaskOnlyOne genGetRegMask(const LclVarDsc* varDsc);
regMaskOnlyOne genGetRegMask(GenTree* tree);
Expand Down
4 changes: 2 additions & 2 deletions src/coreclr/jit/codegenlinear.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -202,8 +202,8 @@ void CodeGen::genCodeForBBlist()
// here. That would require handling the changes in recordVarLocationsAtStartOfBB().

AllRegsMask newLiveRegSet;
regMaskGpr newRegGCrefSet = RBM_NONE;
regMaskGpr newRegByrefSet = RBM_NONE;
regMaskGpr newRegGCrefSet = RBM_NONE;
regMaskGpr newRegByrefSet = RBM_NONE;
#ifdef DEBUG
VARSET_TP removedGCVars(VarSetOps::MakeEmpty(compiler));
VARSET_TP addedGCVars(VarSetOps::MakeEmpty(compiler));
Expand Down
22 changes: 11 additions & 11 deletions src/coreclr/jit/codegenxarch.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9077,8 +9077,8 @@ void CodeGen::genEmitHelperCall(unsigned helper, int argSize, emitAttr retSize,

emitter::EmitCallType callType = emitter::EC_FUNC_TOKEN;
addr = compiler->compGetHelperFtn((CorInfoHelpFunc)helper, &pAddr);
regNumber callTarget = REG_NA;
AllRegsMask killMask = compiler->compHelperCallKillSet((CorInfoHelpFunc)helper);
regNumber callTarget = REG_NA;
AllRegsMask killMask = compiler->compHelperCallKillSet((CorInfoHelpFunc)helper);

if (!addr)
{
Expand Down Expand Up @@ -9721,7 +9721,7 @@ void CodeGen::genOSRRecordTier0CalleeSavedRegistersAndFrame()
// Figure out which set of int callee saves was already saved by Tier0.
// Emit appropriate unwind.
//
PatchpointInfo* const patchpointInfo = compiler->info.compPatchpointInfo;
PatchpointInfo* const patchpointInfo = compiler->info.compPatchpointInfo;
regMaskGpr tier0IntCalleeSaves = patchpointInfo->CalleeSaveGprRegisters() & RBM_OSR_INT_CALLEE_SAVED;
int const tier0IntCalleeSaveUsedSize = genCountBits(tier0IntCalleeSaves) * REGSIZE_BYTES;

Expand Down Expand Up @@ -9800,7 +9800,7 @@ void CodeGen::genOSRSaveRemainingCalleeSavedRegisters()

// Figure out which set of int callee saves still needs saving.
//
PatchpointInfo* const patchpointInfo = compiler->info.compPatchpointInfo;
PatchpointInfo* const patchpointInfo = compiler->info.compPatchpointInfo;
regMaskGpr tier0IntCalleeSaves = patchpointInfo->CalleeSaveGprRegisters() & RBM_OSR_INT_CALLEE_SAVED;
unsigned const tier0IntCalleeSaveUsedSize = genCountBits(tier0IntCalleeSaves) * REGSIZE_BYTES;
regMaskGpr const osrIntCalleeSaves = rsPushRegs & RBM_OSR_INT_CALLEE_SAVED;
Expand Down Expand Up @@ -10135,13 +10135,13 @@ void CodeGen::genFnEpilog(BasicBlock* block)
PatchpointInfo* const patchpointInfo = compiler->info.compPatchpointInfo;

regMaskGpr const tier0IntCalleeSaves = patchpointInfo->CalleeSaveGprRegisters() & RBM_OSR_INT_CALLEE_SAVED;
regMaskGpr const osrIntCalleeSaves = regSet.rsGetModifiedGprRegsMask() & RBM_OSR_INT_CALLEE_SAVED;
regMaskGpr const allIntCalleeSaves = osrIntCalleeSaves | tier0IntCalleeSaves;
unsigned const tier0FrameSize = patchpointInfo->TotalFrameSize() + REGSIZE_BYTES;
unsigned const tier0IntCalleeSaveUsedSize = genCountBits(allIntCalleeSaves) * REGSIZE_BYTES;
unsigned const osrCalleeSaveSize = compiler->compCalleeRegsPushed * REGSIZE_BYTES;
unsigned const osrFramePointerSize = isFramePointerUsed() ? REGSIZE_BYTES : 0;
unsigned const osrAdjust =
regMaskGpr const osrIntCalleeSaves = regSet.rsGetModifiedGprRegsMask() & RBM_OSR_INT_CALLEE_SAVED;
regMaskGpr const allIntCalleeSaves = osrIntCalleeSaves | tier0IntCalleeSaves;
unsigned const tier0FrameSize = patchpointInfo->TotalFrameSize() + REGSIZE_BYTES;
unsigned const tier0IntCalleeSaveUsedSize = genCountBits(allIntCalleeSaves) * REGSIZE_BYTES;
unsigned const osrCalleeSaveSize = compiler->compCalleeRegsPushed * REGSIZE_BYTES;
unsigned const osrFramePointerSize = isFramePointerUsed() ? REGSIZE_BYTES : 0;
unsigned const osrAdjust =
tier0FrameSize - tier0IntCalleeSaveUsedSize + osrCalleeSaveSize + osrFramePointerSize;

JITDUMP("OSR epilog adjust factors: tier0 frame %u, tier0 callee saves -%u, osr callee saves %u, osr "
Expand Down
Loading