Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run on hardware iOS device #95

Open
saikrishna321 opened this issue Mar 7, 2017 · 77 comments
Open

Run on hardware iOS device #95

saikrishna321 opened this issue Mar 7, 2017 · 77 comments

Comments

@saikrishna321
Copy link

No description provided.

@LeoNatan
Copy link
Contributor

LeoNatan commented Apr 4, 2017

There are several challenges we need to overcome before we can achieve this.
Please know there there will be some limitations to what will be possible on the device, such as permission changing.

@silyevsk silyevsk self-assigned this May 8, 2017
@simonracz
Copy link
Contributor

simonracz commented May 10, 2017

Here is how we achieved this recently.

  • We took the "Copy Detox Framework" run script from the demo-react-native example project.
  • As I ran into code signing issues, I replaced it by manually embedding the Detox.framework into the project.
  • I added a new script that recursively code signs each framework and dylib inside Detox.framework.
  • I copied and embedded the XCTest.framework also in the project as Earlgrey was complaining about it.
  • I set up the detoxServer and detoxSessionId in the launch arguments of the scheme.
  • The device and laptop was on the same network of course. The laptop's IP was set up correctly in the appdelegate and in the detoxServer launch argument.
  • I set up the same port and sessionId in the detox config (in package.json)
  • Also added an "ios.none" config there.
  • I ran "detox run-server" manually
  • Then, I ran "detox test --configuration ios.none".

And now, about the problems. After setting these all up, the app was running, it connected to the server, the tests started to run. And each of them failed. :|

During investigating what could be wrong, it suddenly started to work. Without any code change. The same pattern repeated on one of my colleague's machine. (It failed the tests first, then it passed them half an hour later.)

@rotemmiz
Copy link
Member

@simonracz did you run a react-native app? in debug mode ? Loading the bundle from the debug node server may take time, and detox may reach the default timeout and fail.

@simonracz
Copy link
Contributor

If you mean the Mocha runner time out, then no, it wasn't that.

We tried both release and debug mode. It was the demo-react-native sample. No modifications were made to it, apart from the project file modifications mentioned above.

The first test line is "await expect(element(by.id('welcome'))).toBeVisible();". This failed with an Earlgrey assertion. It found the view with the id welcome, however it failed the visibility test.

I started to debug the runtime view hierarchy. For example, I stopped the process and called "po [GREYElementHierarchy hierarchyStringForAllUIWindows]".

For this view it showed the AX.id correctly, but the AX.frame was (0,0),(0,0). Sadly, I don't have saved logs.

After it started to work, the AX.frame was correct.

I wasn't able to reproduce it since then.

@simonracz
Copy link
Contributor

simonracz commented Jul 4, 2017

I was able to reproduce this "Earlgrey fails the visibility test" issue.

In fact I spent half-a-day on chasing this. I was able to narrow it down to the following.

The matched view is rejected, because it's accessibilityFrame is zero. It's frame is correct. It is an RCTView, which inherits from UIView. From to the docs "The default value for this property is CGRectZero unless the receiver is a UIView object or is a subclass of UIView, in which case the value is the frame of the view."

So, this shouldn't happen unless someone manually touches the accessibilityFrame. I looked at the react-native source-code, they never touch the accessibilityFrame. Neither does Earlgrey (except in it's tests).

I restarted, cleared, rebuilt, killed everything. Nothing helped.

I don't think this is a Detox issue. I wrote it down only that in case you encounter it, let's share our findings.

@simonracz
Copy link
Contributor

After spending more than a day on this bug I found the issue.

Since some iOS version (don't know yet exactly when, but 10.3.2 and 10.3.3 are affected) Apple introduced a bug/feature around accessibility.

The accessibilityFrame is not set, nothing isAccessible etc. unless the user is really using some accessibility features, e.g. voiceover.

Earlgrey heavily depends on accessibility. I don't know why haven't anyone else found this, opened a bug about it yet.

I'll open a bug report on Earlgrey and also open radar. I want to determine first the exact iOS version when this was introduced.

@LeoNatan
Copy link
Contributor

LeoNatan commented Jul 5, 2017

@simonracz
Interesting. XCUITest also uses accessibility for its operation. It’s worth investigating how the system allows that for those cases but doesn’t for ours. Most likely an env var.

@simonracz
Copy link
Contributor

Made a radar about it. http://www.openradar.me/radar?id=5043342748418048

@LeoNatan
Copy link
Contributor

LeoNatan commented Jul 5, 2017

@simonracz
This is probably an intentional change. There is no need to calculate these values on a real device, running production software (I don't mean a release build). We need to find out how the system differentiates between running from Xcode and otherwise. I will take a look soon.

@LeoNatan LeoNatan changed the title [feature] can we run on real device iOS Run on real device iOS Jul 5, 2017
@LeoNatan
Copy link
Contributor

LeoNatan commented Jul 5, 2017

I have found a way to force accessibility.
I will push soon to Detox.

LeoNatan added a commit that referenced this issue Jul 5, 2017
@saikrishna321
Copy link
Author

@LeoNatan is it possible to run on real device now ? I couldn't find any docs on it

@pachun
Copy link
Contributor

pachun commented Sep 13, 2017

@LeoNatan This will be so helpful to have! Thanks for putting in the work. Can't wait to get my hands on it.

@LeoNatan
Copy link
Contributor

Running on devices is not supported yet.

@LeoNatan
Copy link
Contributor

@pachun Please do not unassign members of the team from issues. Leave the issue management to us. Thanks.

@pachun
Copy link
Contributor

pachun commented Sep 14, 2017

@LeoNatan I have no idea how that happened. Sincerest apology. 100% unintentional. All I meant to do was thank you.

@LeoNatan
Copy link
Contributor

Cheers!
We plan to support Detox on device soon.

@sonianin
Copy link

@LeoNatan, are there any updates on the iOS real devices support? We are really looking forward to this functionality...

@LeoNatan
Copy link
Contributor

Nothing to announce yet. It’s in our backlog and we intend to investigate this eventually, but it’s not on the top of our priorities. There are some challenges that we haven’t solved yet.

@mike623
Copy link

mike623 commented Dec 30, 2017

Really looking forward of this too. My app need turn on the real device to check the camera.

@fdnhkj
Copy link
Contributor

fdnhkj commented Jan 8, 2018

@LeoNatan Don't you think the following note in README should be adapted then?

Runs on Devices: Gain confidence to ship by testing your app on a device/simulator just like a real user.

IMO, it is a bit misleading. I would mention it is not yet supported on iOS.

@DanielMSchmidt
Copy link
Contributor

@fdnhkj I think you are right, could you go ahead and make a PR?

@fdnhkj
Copy link
Contributor

fdnhkj commented Jan 8, 2018

@DanielMSchmidt Sure I can!

Do you know if running on physical device on Android works? I have seen android.attached config in the docs but I didn't test it so I'm not sure.

I'll adapt the README accordingly.

@DanielMSchmidt
Copy link
Contributor

DanielMSchmidt commented Jan 8, 2018 via email

@lawrencelomax
Copy link

@LeoNatan these are indeed similar constraints to what idb has to work with.

Essentially the constraints about going through a debugserver to get anything done w.r.t app launching on devices holds on iOS Devices.

On the simulator we are able to load the Detox dylib from any folder on the Mac machine—this is not possible on iOS obviously, so need to be able to somehow send the Detox dylib somewhere accessible within the app sandbox

If the detox lib is part of the App bundle, then you are able to dynamically link it, even using preloads like DYLD_INSERT_LIBRARIES, it's essentially a directive to dyld like any linker directive in the main app image.

The regular use-case here is splitting up an application into a smaller main executable that is loaded on launch and then other dylibs for features within an application. This is used to improve launch time so that that only the main dylib needs to be loaded when the app is launched and additional features can be linked only when they are needed.

@LeoNatan
Copy link
Contributor

LeoNatan commented May 14, 2019

@lawrencelomax We decided to go that route to ease the setup and potential deployment issues of our users. They don't change their project and the system automagically adds the required Detox code in runtime.

Once we are able to sign applications with Detox for running on device (in order to add the Detox dylib inside the bundle), it will be trivial to modify the signed entitlements to allow debugger attach.

@maxkomarychev
Copy link
Contributor

maxkomarychev commented May 20, 2019

hey @LeoNatan I was wondering, how big are changes which you are trying to save users from in this

We decided to go that route to ease the setup and potential deployment issues of our users. They don't change their project and the system automagically adds the required Detox code in runtime.

scenario?

There's amazing xcodeproj project which lets you do pretty much anything you want with xcode project.

So is there another problem on they way if we can implement a solution which manipulates over user's project in the cloud w/o bothering a user to configure detox manually and then maintain parallel versions of the app (with and w/o detox libs).

update

user's project in the cloud w/o bothering a user to configure detox manually and then maintain parallel versions of the app

*or actually before uploading to the cloud

@LeoNatan LeoNatan changed the title Run on real device iOS Run on hardware iOS device May 20, 2019
@LeoNatan
Copy link
Contributor

@maxkomarychev Yes, I am familiar with that project, thank you. We used to use xcodeproj before settling to the current system. In some ways, it was terrible (not xcodeproj, but our previous solution). Detox would make a change in the user's project; since a lot of our users are ... "Xcode challenged" (to be as politically correct as I possibly can in this late hour), they would not understand the changes, and revert some or make changes of their own. We used to have problems with people complaining about changes. Changes done using xcodeproj were hard to change if we wanted to make a change in Detox. Big mess. We won't be going back to that system, sorry.

In your cloud proposed method, signing would still be an issue. We could sign it with an internal certificate, but that cannot guarantee a 100% functioning app (for example, is an app uses iOS extensions, using application groups and accessing files in the group container, etc.). Also, since Detox does not have a centralized backend, every user that would like to run Detox on a device would require them setting up this cloud on their own. I find it easier to just do everything locally (including signing with their own certificate).

@pvinis
Copy link

pvinis commented Oct 14, 2019

Is there any work happening on this one?

@LeoNatan
Copy link
Contributor

No work, sorry.
We are working on rewriting large portions of the project, which should in theory make running on hardware device easier.

@pvinis
Copy link

pvinis commented Oct 14, 2019

Is this happening in open source or is it more of an internal wix effort?

@LeoNatan
Copy link
Contributor

Detox is an open source project, so the changes will be available once ready.

@fjmorant
Copy link

fjmorant commented Feb 3, 2020

Hi,

How is this feature going? Do you have any updates?

Thanks

@LeoNatan
Copy link
Contributor

LeoNatan commented Feb 3, 2020

This is not a priority for us. No updates right now.

@fjmorant
Copy link

fjmorant commented Feb 3, 2020

But it is part of the roadmap

https://github.com/wix/Detox/blob/master/docs/More.Roadmap.md#ios-physical-device-support

is there any plan to make it available at some point?

Thx

@rotemmiz
Copy link
Member

rotemmiz commented Feb 3, 2020

Referencing @LeoNatan's answer #95 (comment)

This is not a priority for us as we still see Simulators as a good enough tool for our testing. The Detox iOS rewrite will make this potentially easier to support, but we'll still need to put some effort into it as a separate task once the rewrite is done.
Re the roadmap, you are right, but it needs a refresh.

@LeoNatan
Copy link
Contributor

LeoNatan commented Feb 3, 2020

The roadmap document isn't really maintained. As part of the rewrite, we need to go over documents and remove some of the more esoteric ones which we don't actively maintain (such as the roadmap one).

@fjmorant
Copy link

fjmorant commented Feb 4, 2020

Hi @rotemmiz and @LeoNatan ,

Thanks for your replies and for giving me a bit of the context with your current situation, I will keep an eye to the latest releases.

Kind Regards

@LeoNatan
Copy link
Contributor

LeoNatan commented Feb 4, 2020

With the pending rewrite, some of the more peculiar deployment issues should be solved (no need for ios-deploy and ability to easily set an environment variable), but some issues will still remain, such as deploying the Detox dylib on device and signing it correctly so it can be loaded on device. Those aren’t particularly hard to solve but require thinking and perhaps additional requirements from the user. Once we finish the rewrite effort, we can revisit this topic.

@TammyTorres
Copy link

@LeoNatan Is not clear for me if till today is possible or not to run on real devices from documentation https://github.com/wix/Detox/blob/master/docs/APIRef.Configuration.md I'm not seeing how to do it

@LeoNatan
Copy link
Contributor

Well, clearly it is not possible, as evident by the discussion here.

@TammyTorres
Copy link

thanks! just want to be sure . Is something that would be possible to provide soon? @LeoNatan

@LeoNatan
Copy link
Contributor

Not currently planned, as stated many times before. We are working on overhauling Detox’ internals for iOS, which will make it easier.

@iradkot
Copy link

iradkot commented Dec 6, 2020

I see the discussion kind of repeat itself, but still, 7 months since last reply, are there any plans on supporting ios physical device in the near future?

@LeoNatan
Copy link
Contributor

LeoNatan commented Dec 7, 2020

Not in the very near future, but we keep this thread open for a reason. We are moving the pieces slowly toward that goal.

@andreialecu
Copy link
Contributor

@LeoNatan I was following #2198 and I just noticed it was merged and published as 18.0.0 but couldn't find any release notes yet.

Does it bring real device support? Can we now run on services like AWS Device Farm?

@LeoNatan
Copy link
Contributor

No. Running on iOS hardware will take a lot more work.

@LeoNatan
Copy link
Contributor

There is no need to ask about this every few months or after every release. The issue is open, and we haven't forgotten about it. Once the work has been done and we are ready to add this support, we will surely write about it here in the issue, as well as in the release notes. Thanks

@wix wix locked and limited conversation to collaborators Dec 29, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests