ImageSegmentation crashes on iPhone and works on simulators

[Gaozhongpai]

I have tested ImageSegmentation on iPhone 13 pro and iPhone 12 pro max. It crashed as follows:

[madhavajay]

I get this error too:

CoreGraphics`ERROR_CGDataProvider_BufferIsNotReadable:
    0x1c2766ee0 <+0>:  sub    sp, sp, #0x10
    0x1c2766ee4 <+4>:  str    x0, [sp, #0x8]
    0x1c2766ee8 <+8>:  mov    w8, #0xaa
    0x1c2766eec <+12>: strb   w8, [sp, #0x7]
    0x1c2766ef0 <+16>: strb   wzr, [sp, #0x7]
    0x1c2766ef4 <+20>: ldr    x8, [sp, #0x8]
->  0x1c2766ef8 <+24>: ldrb   w8, [x8]
    0x1c2766efc <+28>: strb   w8, [sp, #0x7]
    0x1c2766f00 <+32>: add    sp, sp, #0x10
    0x1c2766f04 <+36>: ret    

[madhavajay]

Okay so I got this working, and i'm not sure if my thinking is exactly correct.

1. I used this PR Issue fixed: dangling pointer that caused by 'UnsafeMutablePointer()' #76
   Issue fixed: dangling pointer that caused by 'UnsafeMutablePointer()'  #76

I guess that prevents the app from crashing due to memory on the way into the TorchModule.
But, with the above fix it still crashed probably because the memory coming out of the TorchModule.

2. I then made changes to TorchModule, to return the NSData instead of the raw c++ memory.
   If my understanding is correct this means that Objective-C will handle retain on the NSData with ARC.
   At that point I just needed to figure out which new Swift method would give me an UnsafeMutablePointer<UInt8> to the NSData which took longer than I expected.

But it works! 🎉
I suspect this can be streamlined more now that I have it working without crashing.

My changes are here.
madhavajay@31d8252

@xta0 any chance the above PR could be merged? And what do you think about this remaining bug? I can open a PR if I know what changes are desired but its definitely broken currently which is a little sad considering these TorchScript 1.11 demos are super cool.

[StuvX]

@madhavajay I don't follow where you are allocating the colours to the buffer for the classes, in your fix you return the data without mapping to the class detection... can you steer me to where you've allocated the class map?

[madhavajay]

@StuvX my code doesnt change that it just changes the return type from the objective-c function:

- (NSMutableData*)segmentImage:(void *)imageBuffer withWidth:(int)width withHeight:(int)height {

The way the classes are assigned is by comparing them and setting one of three color channels RGB in the image if they are the max:

buffer[n] = 0; buffer[n+1] = 0; buffer[n+2] = 0;
if (maxi == PERSON) buffer[n] = 255;
else if (maxi == DOG) buffer[n+1] = 255;
else if (maxi == SHEEP) buffer[n+2] = 255;

[StuvX]

@madhavajay Forgive me, my objective-c is not great. you also change 'segmentImage' to return 'data', but your call assigns the colours to 'buffer' - is it because buffer is a pointer to data, and the update call (e.g,. buffer[n] = 255) is updating data[n]?

[madhavajay]

@StuvX Yes, its the same memory so I left the C code as it is, and just made sure that the return type is the NSObject so that Objective-C manages the memory and its survival after the function ends using ARC.

[cyrillkuettel]

@madhavajay Thank you so much! You just made my day.
I've been trying to get the ImageSegmentation to work for hours. Now it works thanks to the fix madhavajay@31d8252