[ARKit] SwiftShot porting issues

[mykyta-bondarenko]

The original sample: https://developer.apple.com/documentation/arkit/swiftshot_creating_a_game_for_augmented_reality
Ported sample: https://github.com/xamarin/private-samples/tree/swift-shot/ios12/SwiftShot

The main 2 problems now:

1. When I am setuping the scene it blinks and disappears. But it's on his place because you can manipulate with it. I can reproduce it in 99% when debugger is attached and in 30% when it's not. I spent a lot of time to find the reason but everything that I found is "not short" background operation. If I comment it - it displays in most cases. I guess that the reason is some invocation that is freezing the thread.
2. The scene is darker that the original one. Just compare two applications and you will see it. I don't know what is causing this behavior

[VincentDondain]

Unfortunately the sample doesn't build. It seems you forgot to commit some files. There should be a Components folder with some files here: https://github.com/xamarin/private-samples/tree/swift-shot/ios12/SwiftShot/SwiftShot/Core/Gameplay%20State.

[VincentDondain]

Hey,

1. So I wasn't able to reproduce issue 1) where the scene blinks and disappears. Maybe I do not understand fully what you're talking about though. Are you talking about the grid that is displayed when you're trying to setup the game board? That grid that you can move and resize? This grid worked fine for me, it only disappears when there aren't enough features and it doesn't detect the plane anymore. That grid also blink in the native sample so I'm wondering if the behavior you're describing is expected.

2. This is the same issue as [tvOS][SceneKit] Darker scene than other platforms #3766 that you filed and was fixed in 0d65ed9. Here's the build:
   https://bosstoragemirror.blob.core.windows.net/wrench/jenkins/master/0d65ed93578d45183b704ccef98f3f939721b9fc/87/package/xamarin.ios-12.3.0.5.pkg

[VincentDondain]

@mykyta-bondarenko-gl can you clarify issue #1 please? Otherwise I guess we can consider this fixed (:

[mykyta-bondarenko]

Sorry for the delay. I made a small video of the issue. Hope you will understand what I meant.
http://www.screencast.com/t/kq0SjVZIML

@VincentDondain ^

[VincentDondain]

Watched the video. So it fails the first time and works the 2nd time. However I'm curious if you can try with the native sample because it doesn't look like a XI specific issue to me.

ARKit requires you to move around the scene to detect feature points in order to attach AR objects to it. Since you didn't move I don't think you detected many feature points.

You might want to try adding scnView.DebugOptions = ARSCNDebugOptions.ShowFeaturePoints | ARSCNDebugOptions.ShowWorldOrigin and see if there's a correlation between the lack of feature points and the issue you're experiencing.

[mykyta-bondarenko]

So it fails the first time and works the 2nd time
but the original one works from the first attempt in 95% cases

Also I had written before - I think that it happens because of some "long" operation. For ex. if I put Task.Delay even into a bacground thread - scene will not appear. To be more precise - it will appear but it will be invisible (I can act with it)

@VincentDondain ^

[VincentDondain]

@mykyta-bondarenko-gl and what happens if you do that with the native sample?

[mykyta-bondarenko]

@VincentDondain if I put Task.Delay into the native sample - behavior will be the same. But in our case debugging is more complicated because of scene is invisible in most cases. Maybe there is some internal operation that run in UI thread but shouldn't or something else...

[radrad]

@mykyta-bondarenko-gl Can you share private-samples/tree/swift-shot/ios12/SwiftShot project. I would like to see how Xamarin would re-implement this game as well as how to port it to Unity?
Thanks

[mykyta-bondarenko]

Hi @VincentDondain
I've found an interesting point and it doesn't look like correct behavior (as for me).
Every time as updateAtTime is invoked I update all components and game object. One of the checks is world position check. I get the PresentationNode property and get needs prop from it.
I've found that If I comment this line all operations become to be much faster. Look at these screenshots pls (1st is the first screen, 2nd is after some game time):
(pay attention to the statistic panel)

• get right world position via accessing PresentationNode property

• the code is commented

You can check it locally with the latest updates. Open RemoveWhenFallenComponent file and comment 23rd line

Checked with: https://gist.github.com/mykyta-bondarenko-gl/7c8d7cd35be4fd1d9cf48fd0c4dc59a0

[VincentDondain]

Ok we clarified the previous comment on slack.

So indeed it seems that there is a big difference between the native and managed sample in "Delegate" time spent (Render Debug Window) which seems to be mainly caused by: https://github.com/xamarin/private-samples/blob/swift-shot/ios12/SwiftShot/SwiftShot/Core/Gameplay%20State/Components/RemoveWhenFallenComponent.cs#L23

Here's a screenshot of the render debug window of the native sample, with the world position code still in place:

I did some profiling (Device|Release) and here are the results:

I see the same Class.LookupClass performance issues reported here: #4936

Otherwise there's a bit of time spent in Xamarin_iOS_ObjCRuntime_Runtime_ConstructNSObject_T_REF_intptr_System_Type_ObjCRuntime_Runtime_MissingCtorResolution_intptr_intptr, see:

Not 100% what this is related to in the code (I have some guesses but I'm not sure).

Invoking @rolfbjarne for to explain both performance diagnostics.

[VincentDondain]

Instruments trace: SwiftShotCSharp.trace.zip

[rolfbjarne]

@VincentDondain can you try my issue-4936 branch and check if the performance is better?

[VincentDondain]

@rolfbjarne will try.

@radrad we'll push the sample to https://github.com/xamarin/ios-samples when it's ready. The repo you're mentioning is our private repo with samples that are in progress (:

[VincentDondain]

@rolfbjarne yea looks much better!

[VincentDondain]

@mykyta-bondarenko-gl So the only remaining issue here is the disappearing scene when debugging.
However I still cannot reproduce that when debugging, tried on 2 machines.

@mykyta-bondarenko-gl can you still reliably reproduce that with stable (or/and dogfood) builds? Could you run instruments on the app when you trigger the issue (maybe we'll see something obvious there)?

[rolfbjarne]

The master branch now has all the performance fixes for #4936.

[radrad]

@rolfbjarne Do you play to publish swiftshot xamarin version soon? What exactly will be the path to that sample?
Thanks

[VincentDondain]

@radrad we are currently still validating the sample (it's a big one) and @mykyta-bondarenko-gl might be able to give you more info about when it will be released.

When done it will be in https://github.com/xamarin/ios-samples.

Also @mykyta-bondarenko-gl I discussed with Sebastien today and we believe that the debugger issue can be due to our soft debugger but there isn't much we can do about it. The debug mode isn't optimized (compared to release for instance, that can explain some of the slowness) and the soft debugger requires extra work that a hard debugger doesn't require (compared to natif).

Given that we've addressed the other issues reported here, since I cannot reproduce the debugger issue and since it's not a problem with the SDK/bindings or something that would affect apps published to the store I'll close this issue (:

[mykyta-bondarenko]

Hi @radrad, it's under documentation team review already. Hope it will be public soon.
@VincentDondain

Thanks

[radrad]

Thanks @VincentDondain.
Should I look under: https://github.com/xamarin/ios-samples/tree/master/ios12 as SwiftShot or since it is Xamarin it will be named XamarinShot?

[VincentDondain]

@radrad yes that's exactly when it will be, might be renamed (:

[mykyta-bondarenko]

Hi @radrad, it was finally merged, you can find it here.

[radrad]

@mykyta-bondarenko-gl Thanks a lot for great work. I noticed some things were not implemented (as noted in the code). Was this due to difficulty or no time? Is this game "almost" 100% equivalent in features and coding as SwiftShot?

[VincentDondain]

There's still at least 1 pending issue #4970 we need time to get to that (there are more pressing things). Otherwise I don't know what exactly you're referring to, this game is definitely equivalent to the native sample (: