Fix offline packaging using event stream #2138

akshita31 · 2018-03-26T21:58:58Z

codecov · 2018-03-26T22:06:54Z

Codecov Report

Merging #2138 into master will increase coverage by 0.05%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master    #2138      +/-   ##
==========================================
+ Coverage    55.7%   55.76%   +0.05%     
==========================================
  Files          64       64              
  Lines        3626     3628       +2     
  Branches      543      544       +1     
==========================================
+ Hits         2020     2023       +3     
  Misses       1440     1440              
+ Partials      166      165       -1

Flag	Coverage Δ
#integration	`50.88% <33.33%> (+0.02%)`	⬆️
#unit	`53.83% <33.33%> (-0.16%)`	⬇️

Impacted Files	Coverage Δ
src/observers/BaseLoggerObserver.ts	`80% <66.66%> (-20%)`	⬇️
src/omnisharp/delayTracker.ts	`73.68% <0%> (+5.26%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3488e2f...ef8c30c. Read the comment docs.

rchande

What's our strategy for not regressing this in the future?

DustinCampbell

looks good!

rchande

Test LGTM, but we should figure out a way to not run this on every build

rchande · 2018-03-29T22:43:00Z

gulpfile.js

@@ -102,7 +102,7 @@ function doPackageSync(packageName) {

 function doOfflinePackage(platformInfo, packageName, packageJSON) {
    if (process.platform === 'win32') {
-        throw new Error('Do not build offline packages on windows. Runtime executables will not be marked executable in *nix packages.');
+        //throw new Error('Do not build offline packages on windows. Runtime executables will not be marked executable in *nix packages.');


Should this get enabled?

rchande · 2018-03-29T22:44:46Z

test/releaseTests/offlinePackage.test.ts

+
+suite("Offline packaging of VSIX", function () {
+    let vsixFiles: string[];
+    this.timeout(2000000);


If this timeout is in milliseconds, that means this test takes nearly 30 minutes to run in travis? Probably not desirable to run that on every build?

rchande · 2018-03-29T22:45:25Z

test/releaseTests/testAssets/testAssets.ts

+import * as cp from 'child_process';
+import * as path from 'path';
+
+export function invokeCommand(args: string[]){


invokeNode? invokeCommand sounds like it could invoke any program

rchande · 2018-03-29T22:50:13Z

@akshita31 Can we add a check that checks if ("TRAVIS_PULL_REQUEST_BRANCH" == """) and then does the release test? That will not run these expensive tests on people's PRs, but will fail releases.

rchande · 2018-03-29T22:50:40Z

@TheRealPiotrP @colombod Thoughts on when we should run this slow and infrequently regressed test?

rchande · 2018-03-29T22:53:53Z

Also @akshita31 in my earlier comment, I was referring to travis.yml :)

rchande · 2018-04-02T19:44:12Z

.travis.yml

@@ -34,7 +34,10 @@ script:
  - npm run cov:instrument
  - npm test --silent
  - npm run cov:report
-  - npm run test:release
+  - npm run test:artifacts
+  - if [ $TRAVIS_PULL_REQUEST_BRANCH == "" ]; then


One other thing: can you also check that travis tag != master? That way we'll only run this when someone starts a release (the release process on GitHub creates a new branch).

TheRealPiotrP

Once travis tag != master is added per @rchande we should merge this.

Convert gulpfile to ts

TheRealPiotrP · 2018-04-02T23:08:58Z

.travis.yml

@@ -34,7 +34,8 @@ script:
  - npm run cov:instrument
  - npm test --silent
  - npm run cov:report
-  - npm run test:release
+  - npm run test:artifacts
+  - test $TRAVIS_TAG != "master" && npm run test:release


Is this sufficient? We don't need to also check for PR build?

rchande · 2018-04-03T15:03:35Z

.travis.yml

@@ -35,7 +35,7 @@ script:
  - npm test --silent
  - npm run cov:report
  - npm run test:artifacts
-  - 'if [ "$TRAVIS_TAG" != "master" && "$TRAVIS_PULL_REQUEST" = "false" ]; then npm run test:release; fi'
+  - 'if [[ "$TRAVIS_TAG" != "master" && "$TRAVIS_PULL_REQUEST" = "false" ]]; then npm run test:release; fi'


Why do we need double brackets?

"&&" operator requires double brackets - https://stackoverflow.com/questions/15993062/bash-scripting-missing/15993083. Without using that I was getting an error

Cool, good to know :)

rchande

LGTM--thanks for setting up the on-release tests.

rchande · 2018-04-03T17:05:29Z

@akshita31 BTW, we should probably "Squash merge"t his one :)

@akshita31

* Initial manual validation plan * Add info about unity * Enable usage of omnisharp.path option for downloading multiple versions of omnisharp (#2028) * Enable usage of multiple versions * Either load the server from a path or download the version packages * Tests for the package creator * Added null check and removed semver check in package creator * Test for the experiment omnisharp downloader * Added test for package manager * Code clean up * Added null or empty check for version * Changes * Modified the description Put the clean up logic in the teardown function * Remove comment * Remove unnecessary usage * CR comments * Removed experimental * Modified launcher * Removed experimental * Modified tests * Modified package description to include version information * Renamed launch path * Add more tests * Changed the description in package.json * "Run All Tests" and "Debug All Tests" (#1961) * Run All Tests Running But Building repeatedly * Disposable variable for run all test * Added code for debug all tests * Code Cleaning * Run all Tests running - Better logs required * Run Tests running, all output shown at the end of all the tests * Renamed variable to methodsInClass * Changes for Debug All Tests * Changes for debug tests request * Debug All Tests running * Added common helpers for single test and all test functions * Improved logs for Run All Tests * Changes to get only 1 response for a set of methods * Extracted a common helper to get the test feature * Resolved review comments * Changes to not show this change for legacy projects * Renamed incorrect variable * Removing foreach for proper order of execution * Remove unnecessary import * Do not show the festure for legacy projects * Make hover test run by adding the file in the project (#2058) * Added hover.cs file * Not requiring server restart * Using Structured Documentation for Signature Help and improving Parameter Documentation (#1958) * Structured Documentation in Signature Help * Code clean up * Using only summary of the documentation * Code clean up * Documentation for parameters showing in signature help * Removing unnecesaary import * Using interploated string and fixed spacing * Parameter Documentation using interpolated text * Added tests * Enable download and usage of latest version of omnisharp (#2039) * Enable usage of multiple versions * Either load the server from a path or download the version packages * Tests for the package creator * Added null check and removed semver check in package creator * Test for the experiment omnisharp downloader * Added test for package manager * Code clean up * Added null or empty check for version * Changes * Modified the description * Put the clean up logic in the teardown function * Remove comment * Remove unnecessary usage * CR comments * Removed experimental * Modified launcher * Removed experimental * Modified tests * Modified package description to include version information * Renamed launch path * Add more tests * Changed the description in package.json * Getting latest version info * Refactored code and added tests * Remove unnecessary using * CR comments * Use common function for latest download * Add new line * Resolve binaries on linux * Copy pacakges by value * Renamed experimentalId and some methods * Moved status item out of method and added version to message * Fix typo (#2066) * Fix year in changelog (#2068) Fixed wrong year in changelog * Formatting for better display structure * Move to const fields in server * Make methods private * Move chai-as-promised to devDependencies * Removed unnecessary function * Insert new debugger with support for Symbol Server and Source Link (#2070) * launch.json schema changes and documentation This has the new launch.json schema to support XPlat symbols * Update debugger packages * Update CHANGELOG.md with debugger change * Modified test description * Update .editorconfig JSON rules (#2063) Last week I ran into yet another time when something added a BOM to one of our json files and broke things. This adds some .editorconfig rules for .json to try and prevent such things. While I was at it, I also wanted to switch the indent size for src/tools/*.json to be '2' so that it is consistent with other json schema files the debugger owns. * Updated to reflect limited Desktop CLR support (#2046) * Updated to reflect limited Desktop CLR support * Fixed grammatical error * Refactor CSharpExtDownloader to use the extracted helper functions (#2072) * Refactor downloader * Renamed method and corrected the position of appendLine * Modified GetStatus and the Status object to append the dispose method * Added settings file * Emphasize test codelens * Add another bullet point * Add more details about signature help and tests codelenses * Fix typo * Update Changelog and test-plan for multiple download of omnisharp (#2082) * Update changelog and test-plan for multiple download of omnisharp * Add mocha+wallaby tests (#2091) * Add mocha+wallaby tests eventually the feature tests should be removed and most of our tests should become unit tests that are runnable from the command line or via wallaby. npm run tdd will enable using mocha's command line tdd capability * Fix `tdd` command * Fix test paths * move to tslint-no-unused-expression-chai The package was already installed, but not used. This makes our tests that use chai.should() pass tslint checks * Add Architecture check in AdapterExectuableCommand (#2094) * Add Architecture check in AdapterExectuableCommand x86 messages only show during the first installation. But it still registeres the vsdbg-ui.exe command. This adds a check so it will error instead of returning the command. * Moving arch check into completeDebuggerInstall * Beta1 -> Beta2 * Remove angle brackets in ChangeLog (#2099) * Add product-wide code coverage + codecov.io integration (#2101) Add product-wide code coverage + codecov.io integration Several new scripts were added: npm run cov:instrument: rebuilds your sources, then instruments them for coverage. Subsequent npm run test will generate coverage data into the .nyc_output directory npm run cov:merge-html: merges all reports from .nyc_output and puts a locally viewable coverage report into coverage directory * Refactor logger, reporter and channel into a unified EventStream using Rx Observable (#2084) * Changes to refactor logging in server * Adding packages * Changes * Remove reporter from CSharpExtDownloader * remove telemtery reporter from server.ts * remove reporter from definitionProvider * Remove reporter from dotnetTest.ts * Debugger Activation + Commands * reduce message types * remove reporter from commands.ts * remove channel from status.ts * Remove reporter & logger from extension.ts * Build issues * Add missing rx dependency * Changed to download progress * Removed using and pass platformInfo * Moved files in observer folder * Renamed the files and added omnisharp channel observer * Remove unnecessary format * Changes in main.ts * Remove channel from global declaration * Preserving the context in onNext invocations * Pulled platformInfo out of server * Remove unnecessary variable * Formatting * Renamed observers * Add mocha+wallaby tests eventually the feature tests should be removed and most of our tests should become unit tests that are runnable from the command line or via wallaby. npm run tdd will enable using mocha's command line tdd capability * Code clean up * Fix `tdd` command * Fix test paths * Add initial DotnetChannelObserver test * Testing the download messages * Remove logger from requestQueue.ts * Fix builds * Use package manager factory * Remove Lines * Remove extra appendLine * Added test for csharp logger and channel * Extracted base class for observers * Test having dependency on vscode * vscode adapter changes * Changes for adapter * Refactored Omnisharp Manager * Moved from interfaces to classes * Renamed onNext to post * Created class EventStream * Removed comment * Added missing break * Added test for Omnisharp Logger * Test for OmnisharpLoggerObserver * Test for telemetry reporter observer * Added test for all the observers * minor nits * Changes * Remove unnecessary imports * remove import * Modified failing test * Make tests pass * Renamed platformInfo * CR feedback * 2 -> 3 * Fix codecov integration (#2108) Fix js->ts coverage map transformation in Travis CI where `node` is not on the path. * enable codecov on unit tests (#2118) * enable codecov on unit tests * rename coverage files for uniform globbing * Validate Indentation for OmnisharpProcess messages (#2119) * Clean up signature help test (#2117) * Added parameter * modifications * Cleaned up tests * Changed test codelens casing (#2077) * CodeCov flags (#2125) Lights up integration test coverage and splits reporting data between unit & integration tests * Update project npm dependencies (#2126) * low-risk package updates * high-risk package updates * disable uninstrumented test pass (#2127) * Fix size (#2142) * 1.15.0-beta4 (#2143) * Verify vsix size before release (#2144) * Verify vsix size before release * Remove istanbul from shipping payload * Use the size appropriately in the release test (#2145) * Fix offline packaging using event stream (#2138) * Creating a new eventStream * invokeNode * Artifact tests * Directing the vsix files to a temp folder * Changes to travis to run release tests on release only * if statement * Convert gulpfile to ts * Add bracket * Clean up the observer tests for better readability (#2157) * CsharpLoggerObserver * dotnetchannelobserver * TelemetryObserver * DebugModeObserver * Correct message in the debug observer * Refactoring status using rxjs (#2133) * hack * Refactored status into 4 separate observers * Resolved warning and information messages * Deleted status.ts * Changes to retain this context * Created fascade for statusbar and texteditor * Subscribe to event stream * Working! * Nits * Mocking warning message * Tried mocking setTimeOut * warning message changes * warning message correct definition * virtual time running * done called multiple time * renamed observer and subject * changes * some changes * refactor^2 * merge conflicts * using rx debounce * Warning Message Observer tests * Remove loadsh.debounce and renamed observer * Move the workspace info invocation to server * Clean up * More test to statusBarObserver * Clean up tests * Fixed ttd * Use vscode commands instead of calling into commands.ts * Added test for information message observer * Tests for status bar obsever * project status observer * Changes to show two observers * some more changes * Lot of questions! * build issues * Remove usings * comments * Remove unnecessary cases * Changes * Remove usings * Dipsose the disposables after the server stop event * Remove the cake thing * Project Status Bar * Clean up the tests * Changed to dcoument * Remove unnecessary functions from the adapter * remove unnecessary change * Remove some changes * changes * Test for server error * Removed comment and modified the initialisation * Empty disposable * Corrected the usage of the disposables * Added comments for debouncer * disposable try * Test the vsix, not the build layout (#2156) The VSCode C# Extension build process follows the VS Code docs and runs tests directly inside of its repo root. This unfortunately gives a false sense of security because bugs can be introduced during VSIX packaging [particularly due to missing content in node_modules or excluded via .vscodeignore]. This change addresses this problem by moving our CI tests to execute the VSIX instead of the build's intermediate artifacts. Specifically: build the vsix unpackage the vsix instrument the unpackaged vsix run tests with VS Code Host pointing to the unpackaged vsix This makes our CI tests ~= to the user's runtime experience and will greatly help us with size reduction efforts. To support this change, I also moved our build system from package.json to Gulp. This makes the build scripts significantly easier to understand, provides intellisense for build scripts, and build-time type checking for their contents. I also strengthened the repo's use of .vscodeignore by creating a copy of the file for each scenario [online packages and offline packages]. The new gulp packaging scripts take advantage of these files to produce packages with predictable contents regardless of when packaging occurs. [small caveat, @akshita31 will be adding a test that validates that net-new content does not start sneaking into the vsix package]. * Enable `noImplicitAny` and `alwaysStrict` in `tsconfig.json (#2159) Along the way, fixed a few bugs that were in place due to implicit anys. Also removed dependency on deprecated vscode API. * tsconfig.json: noUnusedLocals, noFallThroughCaseInSwitch, tslint.json: promise-function-async (#2162) Adds noUnusedLocals to tsconfig.json to keep our sources clean Adds noFallThroughCaseInSwitch in tsconfig.json to prevent unintended switch behavior Adds promise-function-async to tslint.json to force all async functions to be marked as async. This is a building block towards eliminating promises in favor of async/await. * Eliminate Thennable<any> (#2163) Provides strong typing for the repo's Thennables. This is a step towards eliminating promises and will provide compiler support for validating that the refactored code has not changed its final return type. * promise to await packaging tasks (#2164) * Remove the status handling from the package manager (#2160) * Changes to remove status from packages.ts * Tests failing * Added tests * Used tooltip and changed flame color * fix merge problems * PR feedback * Removed implicit any * Changes * Update the links to the Source Link spec (#2170) This updates the links we had to the Source Link spec to point at the more official version. * Make the server and the downloader rely on the vscode adapter (#2167) * Moving fakes to testAssets * Change the import for fakes.ts * Test Assets in feature tests * Remove vscode dependency from server.ts * Make the downloading stuff use vscode adapter * Remove vscode as a property * make test package json a field * Run release test only on release (#2172) * Show the channel on download start (#2178) * Add "Launch Unit Test" option (#2180) * unify rx usage to rxjs (#2168) The current codebase uses both rx [v4] and rxjs [v5] implementations. This PR consolidates our use of rx onto the v5 libraries. * Fix disposable (#2189) * Divide the package manager into separate components (#2188) * Feature tests running with refactored package manager * Refactoring packages-1 * Test for the downloader running using the mock server * Using network settings * Changing the package path * Dividing packages.ts into separate components * use filter async * Use tmpfile interface * Check for the event stream * Using FilePathResolver * Remove using * Testing for normal and fallback case working * Resolve the paths * Remove failing case * package installer test-1 * Add package installer test * Create tmp asset * Package Downloader test refactored * Rename files * resolve compile issues * Clean up installer * Clean up the tests * Rename packages * Package installer test * PR feedback * Package Filter test * Remove yauzl.d.ts * Filter test * Added test for invalid zip file * method for getting the request options * remove imports * please resolve the path * remove latest in settings * Package Manager test executing * Use free port in package manager test * Package Manager (using a https server running) * using http mock server * Downloader test running using free port * Update ChangeLog for codelens and status bar (#2205) * PR feedback * Introduce OmniSharpLaunchInfo type * Fix casing of 'OmniSharp' in user-facing error message * Return both 'LaunchPath' and 'MonoLaunchPath' from OmniSharpManager * move back to latest for O# release * Refactor launcher and ensure that we launch on global Mono if possible * Improve coverage a bit * Use async/await in OmniSharpServer._start() * Fix test * Update debugger links for 1.15.0-beta5 This updates the debugger packages to the latest corelr-debug which is now built on .NET Core 2.1 Preview 2. * Update debugger changelog and debugger-launchjson.md for 1.15 (#2230) - Add debugger items to the changelog - Update debugger-launchjson.md to reference the new launchSettings.json feature. * Mark the C# extension as non-preview (#2220) * Mark the C# extension as non-preview This changes the branding on the C# extension so that it is no longer labeled a 'preview'. Two changes: 1. Change `preview` to `false` in package.json 2. Update the license that is used in official builds of the C# extension. This new EULA is no longer a pre-release EULA and it also has the latest text. * Update README.md as well

akshita31 added 3 commits March 26, 2018 11:41

Creating a new eventStream

2bb2962

fixed

e829569

removeComment

a3199f5

akshita31 requested review from rchande and DustinCampbell March 26, 2018 22:04

rchande suggested changes Mar 26, 2018

View reviewed changes

akshita31 requested review from TheRealPiotrP and colombod March 26, 2018 22:30

DustinCampbell approved these changes Mar 27, 2018

View reviewed changes

colombod approved these changes Mar 27, 2018

View reviewed changes

akshita31 added 4 commits March 29, 2018 10:35

Merge remote-tracking branch 'upstream/master' into fixOffline

dcadd23

Invoke the node command

4055740

some tests

07ed7aa

Added test for offline packaging

5e79909

rchande suggested changes Mar 29, 2018

View reviewed changes

remove change in gulpfile

3ca01c6

akshita31 added 4 commits March 29, 2018 16:28

invokeNode

761dd54

Artifact tests

de7d8da

Directing the vsix files to a temp folder

bc78964

Changes to travis to run release tests on release only

0966fc1

rchande approved these changes Apr 2, 2018

View reviewed changes

if statement

fcbdc5e

rchande reviewed Apr 2, 2018

View reviewed changes

TheRealPiotrP approved these changes Apr 2, 2018

View reviewed changes

Piotr Puszkiewicz and others added 2 commits April 2, 2018 12:57

Convert gulpfile to ts

2e8e647

Merge pull request #7 from TheRealPiotrP/fixOffline

0eeb691

Convert gulpfile to ts

akshita31 added 3 commits April 2, 2018 13:27

check travis tag

9aee396

Test trial

649cb45

C'mon travis!

fb3ae4c

TheRealPiotrP reviewed Apr 2, 2018

View reviewed changes

akshita31 added 6 commits April 2, 2018 16:20

Try try!

4abeeba

Using bash ne

ce7d68e

Add newline

cfbf862

Enclosed in quotes

197fc85

Try travis

e206c9b

Do on false

1006877

akshita31 force-pushed the fixOffline branch from c044ad5 to 1006877 Compare April 3, 2018 02:19

Add bracket

ef8c30c

rchande reviewed Apr 3, 2018

View reviewed changes

rchande approved these changes Apr 3, 2018

View reviewed changes

akshita31 merged commit 4719181 into dotnet:master Apr 3, 2018

akshita31 deleted the fixOffline branch April 3, 2018 17:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix offline packaging using event stream #2138

Fix offline packaging using event stream #2138

akshita31 commented Mar 26, 2018

codecov bot commented Mar 26, 2018 •

edited

Loading

rchande left a comment

DustinCampbell left a comment

rchande left a comment

rchande Mar 29, 2018

rchande Mar 29, 2018

rchande Mar 29, 2018

rchande commented Mar 29, 2018

rchande commented Mar 29, 2018

rchande commented Mar 29, 2018

rchande Apr 2, 2018

TheRealPiotrP left a comment

TheRealPiotrP Apr 2, 2018

rchande Apr 3, 2018

akshita31 Apr 3, 2018

rchande Apr 3, 2018

rchande left a comment

rchande commented Apr 3, 2018

Fix offline packaging using event stream #2138

Fix offline packaging using event stream #2138

Conversation

akshita31 commented Mar 26, 2018

codecov bot commented Mar 26, 2018 • edited Loading

Codecov Report

rchande left a comment

Choose a reason for hiding this comment

DustinCampbell left a comment

Choose a reason for hiding this comment

rchande left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rchande commented Mar 29, 2018

rchande commented Mar 29, 2018

rchande commented Mar 29, 2018

Choose a reason for hiding this comment

TheRealPiotrP left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rchande left a comment

Choose a reason for hiding this comment

rchande commented Apr 3, 2018

codecov bot commented Mar 26, 2018 •

edited

Loading