Jitted Code Pitching Support

[sergign60]

We propose to support releasing memory that is allocated for methods that have been executed and will never be executed again or can be recompiled. That is needed for devices with limited memory capacity.

Also when GDB JIT feature (#6278) is enabled there is additional memory consumption (~700 bytes on ARM32 per method with several lines) for ELF debugging information. Such memory can be released when jitted code for method itself released.

The existing execution model of coreclr without releasing memory for executed code and debugging info creates serious obstacles for applications running on devices with limited memory or even can make impossible their execution in certain cases.

@papaslavik @brucehoult @Dmitri-Botcharnikov @leemgs @lemmaa @wateret

[jkotas]

The earlier versions of CLR had this mechanism. It was called code-pitching but it was removed because of it was very unreliable. The code for it was part of "Shared Source Common Language Infrastructure" that you can find around if you would like to take a look. The hard problem with code pitching is to guarantee that the method being discarder is not executing, and that nobody is holding pointer to it.

[jkotas]

cc @noahfalk @dotnet/jit-contrib

[sergign60]

Is it possible to define that the method is not executing by stopping the program execution and inspection of stack frames for all threads?

[jkotas]

Yes, the original code pitching implementation have done something similar.

[sergign60]

except for FCALLs? Because I read in comments
// FCall is a high-performance alternative to ECall. Unlike ECall, FCall
// methods do not necessarily create a frame. Jitted code calls directly
// to the FCall entry point. It is possible to do operations that need
// to have a frame within an FCall, you need to manually set up the frame
// before you do such operations.

[jkotas]

FCalls are a problem because they are implemented in C++ or hand-written assembly, and so they cannot be discarder.

[jkotas]

needed for devices with limited memory capacity

It may be a good idea to do analysis on how much memory can be saved by doing this for the typical applications. The JITed code does not typically dominate the working set.

[papaslavik]

For data we have garbage collection... So it sort of has a control on the memory increase for data.

@jkotas I wonder if you mean the meta-data more than just data compared to the code here...

[papaslavik]

Also it's interesting if it's possible to write a test using serialization an de-serialization or something like that which causes an unbounded increase in the memory for the code of dynamic methods.

[papaslavik]

The hard problem with code pitching is to guarantee that the method being discarder is not executing, and that nobody is holding pointer to it.

Do you mean the backpatching thing here or arbitrary pointers to the code?

[papaslavik]

I also believe such a feature could be useful for the application of an aggressive inlining etc.

[papaslavik]

@jkotas From my impression on average the JITed code size of a small function on ARM is 3 times larger than the corresponding IL assembly of the method (not talking about too small functions).

Of course, the functions which have a lot of calls interfered with the local variables might have the factor of 8 or similar.

[papaslavik]

E.g. if we take a test JIT/SIMD/CircleInConvex_r/CircleInConvex_r.exe:

Method ClassLibrary.test:convex_hull(ref) has 476 IL size and 1426 emitted size
ClassLibrary.test:Check(ref):bool : 166 IL and 570 emitted
A small function ClassLibrary.test:abs(float):float has 19 IL vs 114 emitted

Also the method Main() for the test IT/Directed/cmov/Float_Xor_Op_cs_d/Float_Xor_Op_cs_d.exe has a bigger factor: 3763 IL vs 33234 emitted

So, I believe there might be a considerably big application which has a lot of executed code.

[jkotas]

First, I do agree that having ability to drop JITed code would be really nice.

Based on my experience, I do not believe that it will move the needle significantly (e.g. more than 1.5x) in total RAM consumption of managed process, or in ability to run on very low end devices, for typical apps.

[papaslavik]

But if you had to have a Facebook and a Browser application on a low-end device, it would be a non-typical but a very resource consuming app.

[papaslavik]

Still Facebook presumably would have a lot of class description meta-data present in the RAM.
So it would nevertheless be about 1.5x as you say, but the absolute number could be significant.

[papaslavik]

@jkotas By the way, is the class unloading feature present in coreclr?

Well, I don't think it would be, provided, no code unloading is supported.

[jkotas]

Class unloading is present for Reflection.Emit (AssemblyBuilderAccess.RunAndCollect). It works with assembly granularity.

[brucehoult]

Method ClassLibrary.test:convex_hull(ref) has 476 IL size and 1426 emitted size

Out of interest, I quickly converted convex_hull(List a) to C++ by adding my own C#-like declarations for Point and List and making a, up, down references (fewer syntax changes than making them pointers). clang++ -Os compiled it to 0x27a (634) bytes of Thumb2 armhf code. Note: cw() and ccw() were both fully inlined. Making all the functions virtual increased it to 0x2d6 (726) bytes.

struct Point {float X; float Y;};

template <class T>
class List{
public:
  List();
  int Count;
  void Add(T);
  void RemoveAt(int);
  void Clear();
  T& operator[](int i);
  T& Last();
  typedef int cmp_fn(T&, T&);
  void Sort(cmp_fn);
};

[gkhanna79]

@sergign60 Is this still an active issue tracking work being done?

[sergign60]

@gkhanna79
CC: @jkotas @Dmitri-Botcharnikov @papaslavik

I'm still working on it My latest results on the CoreCLR test suite are

=======================
     Test Results
=======================
# CoreCLR Bin Dir  : ./coreclr/bin/Product/Linux.x64.Checked
# Tests Discovered : 7028
# Passed           : 6758
# Failed           : 3
# Skipped          : 267
=======================

dropped methods:  1970719
dropped bytes:  724175359
jitted bytes:  1226354249
Percentage (dropped of jitted code) : 59.05%

=======================
     Test Results
=======================
# CoreCLR Bin Dir  : ./coreclr/bin/Product/Linux.x64.Debug
# Tests Discovered : 7028
# Passed           : 6756
# Failed           : 5
# Skipped          : 267
=======================

dropped methods:   2688876
dropped bytes:  1115856311
jitted bytes:  2093793657
Percentage (dropped of jitted code): 53.29%

But I still have five tests with parallel threads fail. I'm looking for reasons of these fails (I believe that there is one reason) You can look at my code in the repository https://github.com/sergign60/coreclr

[sergign60]

dotnet/coreclr#10496