Sort import packages by std lib first #2803

karthikraobr · 2019-10-02T11:57:57Z

This PR partially resolves #2683. The list is sorted by std lib packages first when there is no user input. But this ordering doesn't work when the user starts entering characters to the import box.
@ramya-rao-a any pointers on how to fix this after the user starts entering characters?

ramya-rao-a

Hey @karthikraobr!
Long time no see :)

But this ordering doesn't work when the user starts entering characters to the import box.
@ramya-rao-a any pointers on how to fix this after the user starts entering characters?

We are using the quick pick control here and that control is responsible for maintaining the order after characters are being entered. I tried your code, and I see the order being maintained.

ramya-rao-a · 2019-10-06T19:37:27Z

src/goImport.ts

@@ -20,10 +20,18 @@ export async function listPackages(excludeImportedPkgs: boolean = false): Promis
 		? await getImports(vscode.window.activeTextEditor.document)
 		: [];
 	const pkgMap = await getImportablePackages(vscode.window.activeTextEditor.document.fileName, true);
-
+	const stdLib = await getStandardLibraryPackages();


Instead of making another call to go list which we know to be expensive, why not update getImportablePackages itself return the information on whether a package is standard or not?

Right now, we use {{.ImportPath}} and {{.Dir}} when calling go list in the goPackages.ts file.
We can add {{.Standard}} to the same call and get a boolean representing what we need

If I am not wrong I guess we are using gopkgs to list the packages and that doesn't support the {{.Standard}} flag. That's the reason I wrote another function that uses go list.

Ah! fair point.

In that case, I would prefer if we can update gopkgs to also support {{.Standard}}
I am guessing that whatever it does under the hood to fetch the packages, it already knows if a package is standard or not and exposing this info is a valid feature request.

Elsewhere in this project in goSuggest.ts file, a use a very crude way to identify the standard packages which is seeing if the import path contains a ".".

So, maybe until gopkgs is fixed, we can use a similar approach?

I will create a PR to support{{.Standard}} in gopkgs and then continue with this PR. That seems to be the cleanest approach right?

@ramya-rao-a fyi uudashr/gopkgs#17

I see that you have used GOROOT in the Dir to determine if the package is standard or not.
We do have access to the {{.Dir}}, so maybe we can use that in the extension, compare with process.env['GOROOT']?

ramya-rao-a · 2019-10-06T19:38:23Z

src/goImport.ts

+			}
+			if (stdLib.get(b)){
+				return 1
+			}


This sorting will only ensure that the standard libraries appear before the others.
But among the standard libraries themselves, we lose the sorting
Similarly, we also lose the sorting among the non standard libraries

karthikraobr · 2019-10-08T20:59:48Z

src/goPackages.ts

+				const [pkgName, pkgPath, pkgDir] = pkgDetail.trim().split(';');
+				if (pkgDir.includes(process.env['GOROOT'])) {
+					stdPkgs.set(pkgPath, pkgName);
+				} else {
+					otherPkg.set(pkgPath, pkgName);
+				}
 			});
-
+			Array.from(stdPkgs.keys()).forEach((val)=>{
+				pkgs.set(val, stdPkgs.get(val));
+			})
+			Array.from(otherPkg.keys()).forEach((val)=>{
+				pkgs.set(val, otherPkg.get(val));
+			})


@ramya-rao-a This looks very crude to me :) esp maintaining two maps and merging them. Could you suggest a way to improve it? Thanks!

Instead of maintaining 2 different maps, we can go back to the single map as before.
That map uses string -> string at the moment
We can change that to string -> PackageInfo where PackageInfo is an interface

export interface PackageInfo { name string; isStd boolean; }

This makes it extendable such that in the future if we want to extract more info from the gopkgs tool, we can do so.

ramya-rao-a · 2019-10-23T05:39:35Z

src/goPackages.ts

+				const [pkgName, pkgPath, pkgDir] = pkgDetail.trim().split(';');
+				pkgs.set(pkgPath, {
+					name: pkgName,
+					isStd: pkgDir.includes(process.env['GOROOT'])


perhaps .startsWith is a better function to use here. We expect the path to start with the goroot right?

Suggested change

isStd: pkgDir.includes(process.env['GOROOT'])

isStd: pkgPath.startsWith(process.env['GOROOT'])

Also, do we want to take care of the case where process.env['GOROOT'] doesn't have any value set? It shouldn't happen so, but if it does, do we might want to default to false here right?

ramya-rao-a · 2019-10-23T05:40:32Z

src/goPackages.ts

-					pkgs.set(pkgPath, pkgName);
+					pkgs.set(pkgPath, {
+						name: pkgName,
+						isStd: pkgPath.includes(process.env['GOROOT'])


This is the case of the old gopkgs. In that case, we will have to treat the output differently right?

I am not entirely sure how the output of {{.Dir}} would look with an earlier version?

https://github.com/tpng/gopkgs is the older tool. Can you give it a try?

It most likely ignores the entire formatting input

Oh sorry. I forgot to try this. Will do it tomorrow.

karthikraobr · 2019-10-30T15:31:19Z

@ramya-rao-a can you please have a look at this? Thanks

ramya-rao-a · 2019-11-04T05:14:42Z

src/goPackages.ts

-					pkgs.set(pkgPath, pkgName);
+					pkgs.set(pkgPath, {
+						name: pkgName,
+						isStd: goroot === null ? false : pkgPath.startsWith(goroot)


pkgPath is not the absolute path. It is the import path. So pkgPath.startsWith(goroot) here for the old gopkgs tool will always return false.

So, we might as well set this to false directly.

I'll soon retire this part of the code as we don't expect anyone to be using the old gopkgs anymore

Or another option is to use this very crude logic I currently use for the auto completion widget

Suggested change

isStd: goroot === null ? false : pkgPath.startsWith(goroot)

isStd: !pkgPath.includes('.')

ramya-rao-a · 2019-11-04T05:35:47Z

I pushed a few commits to this PR @karthikraobr

Simplify the sort logic. Your approach required accessing the map for every 2 entries that gets compared in the sort() algorithm. So pulled that part out.
Tweaked the case of the old gopkgs scenario to use a crude way of figuring out standard packages that I use elsewhere in goSuggest.ts file
Fixed linting errors

Sort import packages by std lib first

6d451d5

ramya-rao-a suggested changes Oct 6, 2019

View reviewed changes

karthikraobr mentioned this pull request Oct 8, 2019

Add support for standard flag uudashr/gopkgs#17

Merged

Use DIR directive to determine if package is part of stdlib

8a8aae9

karthikraobr commented Oct 8, 2019

View reviewed changes

karthikraobr added 2 commits October 22, 2019 22:22

Store stdlib inforrmation from gopkg

e8d97af

Merge master

f316b68

karthikraobr requested a review from ramya-rao-a October 22, 2019 20:26

ramya-rao-a reviewed Oct 23, 2019

View reviewed changes

karthikraobr added 3 commits October 23, 2019 12:08

Handle goroot not set

9288947

Fi sorrting

6e32f8b

Merge master

4e4ee62

karthikraobr requested a review from ramya-rao-a October 25, 2019 22:21

ramya-rao-a reviewed Nov 4, 2019

View reviewed changes

ramya-rao-a added 3 commits November 3, 2019 21:31

Improve sorting perf

926b291

Cater to old gopkgs tool

f4bdf41

Fix linting issues

29790f4

ramya-rao-a approved these changes Nov 4, 2019

View reviewed changes

ramya-rao-a merged commit e7e56ea into microsoft:master Nov 4, 2019

karthikraobr deleted the fix/imports-stdlib-first branch November 7, 2019 09:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sort import packages by std lib first #2803

Sort import packages by std lib first #2803

karthikraobr commented Oct 2, 2019

ramya-rao-a left a comment

ramya-rao-a Oct 6, 2019

karthikraobr Oct 7, 2019

ramya-rao-a Oct 7, 2019

karthikraobr Oct 7, 2019

karthikraobr Oct 7, 2019

ramya-rao-a Oct 8, 2019

ramya-rao-a Oct 6, 2019

karthikraobr Oct 8, 2019

ramya-rao-a Oct 20, 2019

ramya-rao-a Oct 20, 2019

ramya-rao-a Oct 23, 2019

ramya-rao-a Oct 23, 2019

karthikraobr Oct 23, 2019

ramya-rao-a Oct 24, 2019

ramya-rao-a Oct 24, 2019

karthikraobr Oct 28, 2019

karthikraobr commented Oct 30, 2019

ramya-rao-a Nov 4, 2019

ramya-rao-a Nov 4, 2019

ramya-rao-a commented Nov 4, 2019

	isStd: pkgDir.includes(process.env['GOROOT'])
	isStd: pkgPath.startsWith(process.env['GOROOT'])

	isStd: goroot === null ? false : pkgPath.startsWith(goroot)
	isStd: !pkgPath.includes('.')

Sort import packages by std lib first #2803

Sort import packages by std lib first #2803

Conversation

karthikraobr commented Oct 2, 2019

ramya-rao-a left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

karthikraobr commented Oct 30, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ramya-rao-a commented Nov 4, 2019