Update production with master to update the setup scripts #3112
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Setup Project Changes Service test suite * Add tests for GetPrepareInfo from ProjectChangesService * Add test to verify changes on a sync file has been applied. * Make ios/android lower case folder names, as it seems like in Linux the case-sensitivity plays a role * Fix platform names casing * Clone the test for Android for file changes * fix lint errors * Add tests for issue #2697 * Adding Tests to verify that the App_Resources have been reloaded * Adding tests for new/deleted platform/common files * Refactor the changes tests and add some more tests * Improve assertions to include message what was wrong with the directory structure * Minor messages improvement * remove delete file tests - seems we do not support this currently. * Adding tests to verify new files in App_Resources are detected and copied to destination. * Remove .only() which was used only for debugging purposes on the Travis CI
* Fix android sdk commands to use sdkmanager * Replace brew install with brew cask * Fix installation for haxm * Change brew formulae repository before installing android sdk * Fix setting the ENV variable missing cast
During `tns debug ios`, in case you make changes, the application must be restarted and the debugger must attached again. However, in many cases we kill the old lldb process and immediately try to start the new one. The childProcess.kill operation finishes, but lldb process does not die immedietely. So in some occasions, the attach of new debugger fails. This leads to multiple errors - you cannot start this application on simulator anymore, you cannot exit CLI's process with `Ctrl + C`, etc. Fix this by attaching to "close" event of the processes and waiting for them to be really finish their execution.
Implement extensibilty of CLI that allows anyone to add easily create packages that add new functionality to NativeScript CLI. The packages are installed in a specific directory, so they are persisted through CLI's updated.
The directory where extensions are installed contains a package.json and each extension is npm package installed there.
The extensions can be mainatined in two different ways:
- navigate to the directory where extensions are installed and use `npm` for install/uninstall/update of packages.
- use CLI's commands to update them: `tns extension install <name>`, `tns extension uninstall <name>`, `tns extension`
Implement extensibilityService that executes all operations and expose it to public API. In {N} CLI the extensions are loaded in the entry point, before parsing command line arguments. This way extensions can add new commands.
In Fusion, after CLI is required as a library, the `extensibilityService.loadExtensions` method should be called. It returns array of Promises - one for each installed extension.
Add help for the new commands, but do not link the new commands in other commands help for the moment.
Add unit tests for the new service.
In case there's no devices attached and no emulators running, trying `<cli name> devices <platform> --available-devices` will start emulator. In order to fix this, modify the `startEmulatorIfNecessary` method to skip the starting in case `skipInferPlatform` option is passed. This option indicates that we are not concerned of specific platform, so the method does not know which is the target platform for which to start emulator. Add unit test for this behavior.
* implicitly install karma-<testingFramework>'s peer dependencies on test init command * add exception handling when installing packages already present in the project * log warning instead of throwing errors when a package's name couldn't be determined when installed using the node-package-manager service
Expose node-package-manager functonality to be usable when CLI is `require`-d. * Remove occurrences of `this.$options` in node-package-manager * Start using `--dry-run` and `--json` for parsing output
When using Chrome DevTools to debug iOS applications, CLI returns a url that points to a specific commit of the dev tools. This way, in case a new version of the dev tools introduces a breaking change, the debugging will still work. However, in some cases (inside Electron app), we cannot use the remote url, so we must use the bundled one. So introduce a new option (only available when requiring CLI as a library), that defines that the returned url will use the bundled dev tools. The default behavior (used in CLI), will still return the url that includes remote url. Also change the definition of `debug` method in the interface, so now the DebugService can safely implement it. Add a new check in the debug service - in case the device's status is not Connected, we are unable to start debug operation. So fail with correct error message in this case. Add JSDocs for all debug related interfaces. Add documentation in PublicAPI.md for the `debugService`. Add debugService to tests of public API.
Add tests for debugService. Add verification that in case device's OS is not supported (for example WP8), we'll throw correct error.
During refactoring we've decided to pass the whole `$options` object from commands to services. However we've forgotten to change the emulate command. Pass correct argument which will fix the issue.
* Fix debug on iOS simulator with watch (#2721) During `tns debug ios`, in case you make changes, the application must be restarted and the debugger must attached again. However, in many cases we kill the old lldb process and immediately try to start the new one. The childProcess.kill operation finishes, but lldb process does not die immedietely. So in some occasions, the attach of new debugger fails. This leads to multiple errors - you cannot start this application on simulator anymore, you cannot exit CLI's process with `Ctrl + C`, etc. Fix this by attaching to "close" event of the processes and waiting for them to be really finish their execution. * Implement extensibility model for CLI (#2724) Implement extensibilty of CLI that allows anyone to add easily create packages that add new functionality to NativeScript CLI. The packages are installed in a specific directory, so they are persisted through CLI's updated. The directory where extensions are installed contains a package.json and each extension is npm package installed there. The extensions can be mainatined in two different ways: - navigate to the directory where extensions are installed and use `npm` for install/uninstall/update of packages. - use CLI's commands to update them: `tns extension install <name>`, `tns extension uninstall <name>`, `tns extension` Implement extensibilityService that executes all operations and expose it to public API. In {N} CLI the extensions are loaded in the entry point, before parsing command line arguments. This way extensions can add new commands. In Fusion, after CLI is required as a library, the `extensibilityService.loadExtensions` method should be called. It returns array of Promises - one for each installed extension. Add help for the new commands, but do not link the new commands in other commands help for the moment. Add unit tests for the new service. * Fix installation scripts for Mac (#2714) * Fix android sdk commands to use sdkmanager * Replace brew install with brew cask * Fix installation for haxm * Change brew formulae repository before installing android sdk * Fix setting the ENV variable missing cast * Do not start emulator when `--available-devices` is passed (#2736) In case there's no devices attached and no emulators running, trying `<cli name> devices <platform> --available-devices` will start emulator. In order to fix this, modify the `startEmulatorIfNecessary` method to skip the starting in case `skipInferPlatform` option is passed. This option indicates that we are not concerned of specific platform, so the method does not know which is the target platform for which to start emulator. Add unit test for this behavior. * Install karma peer dependencies on `test init` (#2693) * implicitly install karma-<testingFramework>'s peer dependencies on test init command * add exception handling when installing packages already present in the project * log warning instead of throwing errors when a package's name couldn't be determined when installed using the node-package-manager service * Freeze mocha-typescript to 1.0.23 (#2746) * Freeze mocha-typescript to 1.0.23 (#2746) * Fix the six package detection in release (#2748)
Vladimirov/merge rel master - we know
Merge release in master
Merge release in master
* Fix getting production dependencies code CLI has logic to find which are the "production" dependencies, i.e. which should be copied from `node_modules` to `platforms` dir. However, when the project has a lot of dependencies (more than 15), on some machines the code leads to error: "Maximum callstack size exceeded". On other machines the code tooks significant time to execute. After investigation, it turned out the recursion inside `node-modules-dependencies-builder` is incorrect and it adds each package many times to the result array. Fix the recursion and change the class NodeModulesDependenciesBuilder to be stateless - instead of using properties in `this` object when calculating the production dependencies, the methods will persist the results through the passed args. This way the whole class can be safely added to `$injector` and used whenever we need the production dependencies. Each time the calculation is started from the beginning, which is the requirement for long living process, where the project may change. Fix the type of the result, which leads to fix in several other services, where the result has been expected as `IDictionary<smth>`. However it's never been dictionary, it's always been an array. The code, that expected dictionary has been working because the `_.values` method of lodash (used all over the places where the incorrect type of data has been expected), returns the same array when the passed argument is array. Fix the tests that incorrectly expected dictionary with keys "0", "1", "2", etc. Remove the usage of Node.js's `fs` module from `NodeModulesDependenciesBuilder` - replace it with `$fs` which allows easir writing of tests. Require the `nodeModulesDependenciesBuilder` in bootstrap, so it can be correctly resolved by `$injector`. Add unit tests for `nodeModulesDependenciesBuilder`. * Use breadth-first search for getting production dependencies Replace the recursion with breadth-first search algorithm in order to make the code easier for understanding and debugging. Fix some incorrect code in the tests. * Add checks before adding new elements to queue of production dependencies Add check before adding new elements to queue of production dependencies - do not add elements, which are already added and do not read package.json of elements that are already added to "resolvedModules".
Passing `--provision` to `tns run <ios>` is not respected. Fix this by including additional option in the deviceDescriptors. This allows us to pass different provisions for each device, however in CLI's commands we'll set one provision for all devices. Also fix passing `--teamId` option - during previous refactoring we've broken the code for it and it has not been respected. Remove the unused alias `teamIdentifier` and use `teamId` in all cases.
Documented `--provision` option in build-ios.md
* Add option to use chrome-devtools fronted from appspot URLs starting with `chrome-devtools://devtools/` cannot be opened in chrome with code. However, we can open the urls in https://chrome-devtools-frontend.appspot.com/, so add new option in Public API that allows using the remote version. Keep the current default behavior, i.e. for Android we are using the `bundled` DevTools, for iOS we are using a commit pointing to Chrome 55. In case both `useBundledDevTools` and `useHttpUrl` are passed, the value of `useBundledDevTools` is disregarded. Add unit tests for all of the cases. * Add `devToolsCommit` to debug options (and public API) Currently CLI uses specific SHA of dev tools, that corresponds to Chrome 55. However, we may need different version for Android or iOS, so add new option. This will also allow specifying it from public API, so external calls may pass the commit SHA.
* Fix stopLiveSync method Currently stopLiveSync method does not emit correct events in some cases. For example, when there's only one attached device and its identifier is passed to `stopLiveSync` method, we do not emit `liveSyncStopped` event. Fix the code and add unit tests to assure `liveSyncStopped` event is raised correctly. * Expose getLiveSyncDeviceDescriptors method Add new public method `getLiveSyncDeviceDescriptors` to `liveSyncService` - its purpose is to get information for current LiveSync operation.
When we prepare the platform for cloud build we skip the native prepare. After this partial prepare, the platform directory contains only .js files. In order to make a local build we should add the native part of the platform which fails with `ENOTEMPTY: directory not empty, rename`.
In case a plugin depends on a scoped package, for example `@angular/core`, trying to use the plugin from local path with npm 5 will fail. The recommended approach is to declare the scoped package (`@angular/core`) as a devDependency of the plugin. After that, declare both the plugin and the scoped package as dependencies of your project. In case the plugin is installed from local directory, npm 5 creates symlink in node_modules. This means that all files inside `<plugins dir>/node_modules` will be visible for CLI, including the dev dependencies. During project preparation CLI copies `node_modules` from the project to `<project dir>/platforms/<platform>...` directory. We have a specific logic to process only dependencies of each package, i.e. we should remove `@angular/core` from the plugin's node_modules direcotry. However, the logic is not working correctly as instead of using the name `@angular/core` it's been trying to remove `core` only. Fix getting names of the dependencies, so we'll be able to remove the scoped packages as well.
…ioningUpdates (#3070) * Setup the provisioningProfiles in the exportOptions plist, provide quiet flags to xcodebuild and gradle, provide -allowProvisioningUpdates to xcodebuild > 9.0 * Using teamId without arg will print all teams, teamId and provision are mutually exclusive, teamId now should work with name if a provision that has the team id for the provided team name is found * Add documentation on --team-id near --provision * Fix versionCompare for the '8.0' and '9.0' versions we compare in ios-project-service * Address PR review * Address another PR comment, native builds will now print short friendly message...
Expose enable/disable debugging when CLI is required as a library. Includes: * Unify `debug-livesync-service` and `livesync-service` into one class. Enable said class to attach a debugger at runtime during LiveSync * Remove redundant `debugLivesync` property from `$config` and its every usage * Remove redundant `debugStop` code in `android-debug-service` * Raise events in the event of inability to start/stop the app on iOS
Due to recent changes in `ios-project-service`, some unit tests cannot work on Windows and Linux, so ignore them in case the OS is not macOS.
Enable `prefer-const` rule in tslint and fix the errors: The rule requires that variable declarations use const instead of let and var if possible. If a variable is only assigned to once when it is declared, it should be declared using `const` Add new npm script for automatic fix of tslint: `npm run tslint-fix`. NOTE: In case you have 3 errors that the script can fix, when you execute `npm run tslint-fix`, it will fix them, but will print them as errors again. Run the command again to verify everything is ok.
* Updated changelog for 3.2.0 RC * Update CHANGELOG.md
…up npm and gradle proxies (#3004)
* Debug improvements Includes but is not limited to: * `enableDebugging` - each promise is now resolved with an object * added docs for `debuggerAttached` event * added `debuggerDetached` event and docs for it * PR Fixes
`debuggerAttached` event should emit full information required for debugging. However, the deviceIdentifier is missing, so add it. Update PublicAPI.md according to latest changes.
* Improve analytics We have a requirement to track information in several different providers. However, this may slow down CLI's operations as each track (at least for some of the providers) may last around 100-300ms. In order to track in multiple analytics provider without slowing down CLI commands, move the tracking to a new process. The process is detached from CLI, so when CLI finishes its work it will exit, while the child process will remain alive until it sends all required information. The new process is called `analytics-broker-process`, it is detached from CLI, but it there's IPC channel between them, so CLI can send information to be tracked. Just before CLI dies, it will send "finish" message to the `broker` process and the IPC channel will be closed. In CLI implement a new `analytics-service`, so when the methods `track`, `trackFeature` and `trackException` are called, instead of tracking directly in CLI's process (i.e. the old implementation), to send information to the broker process. The only exception is tracking of the answer if user allows us to send anonymous statistics. As this action requires user interaction, it is executed in CLI's process, a new eqatec analytics monitor is started and the answer is sent to it. After that the analytics is stopped. In case the answer is "yes" and there is more information that we want to track in the current process, it will be send to broker and tracked in all providers. In case the answer is "no", no other data will be tracked. The broker process is started from analytics process once per process, i.e. on each CLI process (which have something to track) there's a corresponding `analytics-broker-process`. Once it is initialized, it sends BrokerReadyToReceive message to CLI and CLI knows that it can start sending tracking information. Initialization phase is creating a new instance of AnalyticsBroker and attaching handlers for `message` and `disconnect` events. The AnalyitcsBroker itself has a single purpose - sends tracking data to all available prociders. Currently the only provider is `eqatecAnalyticsProvider`. Once CLI sends "finish" message (or `analytics-broker-process` receives `disconnect` event), the process sends information to all providers that there's nothing more to be tracked. When all providers finish their work, they should disocnnect themselves from the broker process and it will exit gracefully. `EqatecAnalyticsProvider` is used to track features and exceptions in Progress Telerik Analytics. Our requirement is to track features in two projects (same data should be send to both of them) and exceptions in another project. Each of them should be in a separate child process, so failure in one of them will not reflect the others. So the provider starts all required processes. When broker sends information to the provider, it sends it to all required child processes, for example if a feature should be tracked, the provider will send information for it to the two processes used for feature tracking. Each eqatec child process is a new Node.js process runnign `eqatec-analytics-process`. Again - this is a detached process. Once it is initialized, it sends "EqatecAnalyticsReadyToReceive" message to the `EqatecAnalyitcsProvider` and it knows it can send information to the new child process. The initialization phase is creation of new instance of `EqatectAnalyticsService`, attaching to "message" and "disconnect" events. Once `EqatecAnalyitcsProvider` receives `EqatecAnalyticsReadyToReceive` message, it sends initialization data for the eqatec monitor. It includes the analytics project API key, user id, current sessions count, etc. The information is persisted in a file on user's machine, so this file should be updated from a single process. As we might have 3 eqatec analytics processes, the provider is the only place where the file should be read and updated. When `eqatec-analytics-process` receives initialization data, it creates a new instance of the `EqatectAnalyticsService` class with it. However a monitor is not started yet. It will be started when there's something to be tracked, i.e. once `eqatec-analytics-process` receives feature or exception to be tracked. When `eqatec-analytics-process` receives "finish" message, it awaits all pending requests and after that disconnets itself from the parent. In case this operation fails, the process just exits, which will allow the parent to exit gracefully. Remove the public `startEqatecMonitor` method from API - it has not been used. * Craete all eqatec analytics monitors in the broker process Eqatec Analytics monitors can be created in the broker process directly instead of starting multiple child processes. So delete some unneeded files and move tracking of `AcceptUsageReporting` out of CLI process. * Add tracking to Google Analytics Add new methods to track information in Google Analytics(GA). As the data that we want to track is different than the one we track in Eqatec Analytics, call the new methods whenever we want to send some information to GA. Change the broker process to send information to Eqatec Analytics when we want to track feature or exception and to GA only when the new method for tracking there is called. In Google Analytics (GA) we track two types of data - pages (called pageviews) and events. Each command is tracked as a page - we only track the name of the command as a page: * `tns build android` will be tracked as visited page - `build android` * `tns run` will be tracked as visited page - `run` When we want to track some additional actions that happen during execution of each command, we send events to GA. Each event has the following data: * category - currently we do not have a specific category for each action, so it is hardcoded to "CLI". * action - this is the real action that will be tracked, for example `Build`, `LiveSync`, `Deploy`, etc. * label - contains specific inforamation for the current action, for example what is the platform, device type, etc. In many cases we send labels with a lot of information, but as the value must be string, we use `_` as a separator. For example: * Action `Build` may have the following label: `Android_Debug_Incremental`. This is `<platform>_<build configuration>_<build type>`. * Action `LiveSync` may have the following label: `Android_Emulator_5.1`. This is `<platform>_<device type>_<device os version>`. Custom dimensions For each additional data may be send to GA via Custom Dimensions. More about them can be found [here](https://support.google.com/analytics/answer/2709828). We are using several custom dimensions, most of them have hit scope, i.e. their values are saved for the current hit (event or page), but may be changed for the next time. The only exclusion is `session id` - it has session scope, so whenever it is sent, the new value overwrites the values of the "session id" property in all hits of the current session. One interesting custom dimension is `Client` - it will be set to the value of `--analyticsClient` in case it is passed. In case not, we'll set it to "CLI" when terminal is interactive and "Unknown" when terminal is not interactive. Sessions - what is session for GA In Eqatec Analytics we call `stopMonitor` at the end of each command. So the session there is one command. In GA we've decided to keep the default way of measuring a session - a session is stopped when there are 30 inactive minutes, i.e. no data is sent from 30 minutes. This allows us to investigate the behavior and the flow of commands a user is executing in a single session (during the day for example). Client ID and User ID: Currently we do not send User ID to GA, we use Client ID instead as sending User IDs requires different logic and agreements. For us all users are anonymous. User-Agent When sending requests to GA, we need to set correct User-Agent, so GA will be able to understand the Operating System. For each OS the string is painfully specific... Current implementation works fine for Windows, Linux and macOS. Implementation In order to send data to GA, we are using a `universal-analytics` npm package. We cannot use Google's default libraries (`analytics.js`, `gtag.js`) as they are written for browser. So the only way to track data is to use the Measurement Protocol (directly send http requests). However, `universal-analytics` package provides a very good abstraction over the Measurement Protocol, so we are using it instead of implementing each call. More information about Measurement protocol: https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters * Fix hanging commands In case the command is not related to LiveSync, once it finishes its work, CLI process should end. However, this works only on Windows, as the broker child process there is really detached from the parent. On macOS and Linux, the 'ipc' channel between the broker process and CLI, keeps both alive. So use the 'dispose' method, which is called at the end of command execution and disconnect the broker process. In case we are in LiveSync case, set shouldDispose to false and this way the broker will be kept alive. Once Ctrl + C is used to end the CLI process, send the finish message to broker and it will disconnect itself from CLI.
Update two of the dependencies to their latest versions. ios-device-lib has a fix that causes hanging in a very rare cases when trying to get installed applications on iOS device: https://github.com/telerik/ios-device-lib/releases/tag/v0.4.9 plist-merge-patch has a fix for merging `LSApplicationQueriesSchemes` values: https://github.com/NativeScript/plist-merge-patch/releases/tag/v0.1.1
rosen-vladimirov
approved these changes
Sep 8, 2017
Move lockfile service to common and lock operation for writing user settings file Add new lockfile service (use lockfile npm package) and lock all operations for reading/writing user-settings.json. These operations may be executed in different processes, so they must be synced between them. Clean obsolete code for lockfile in services that do not longer use it. Ensure failures in eqatec analytics will not prevent tracking in google analytics by try-catching execution in broker service.
|
run ci |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.