Skip to content
This repository has been archived by the owner on Oct 27, 2021. It is now read-only.

Not completing external packages #32

Closed
theSuess opened this issue Jun 9, 2018 · 30 comments
Closed

Not completing external packages #32

theSuess opened this issue Jun 9, 2018 · 30 comments

Comments

@theSuess
Copy link

theSuess commented Jun 9, 2018

I switched over to this fork as nsf/gocode wasn't working at all after go 1.10. Local completion works fine, as does the standard library but as soon as i try to complete an external library or type (e.g github.com/sirupsen/logrus) gocode tells me that 0 candidates were found. I tried a complete clean install but nothing seems to work.

@cfloa
Copy link

cfloa commented Jun 9, 2018

I'm having the same issue, how frustrating.

@jaytaylor
Copy link

Are you using vim? If so, be certain you followed the installation steps precisely to the letter.

@RelicOfTesla
Copy link

RelicOfTesla commented Jun 12, 2018

I'm having the same issus.
this gocode only support some system library on GO 1.10.1.
${go.gopath}/src/main/aaaa.go
${go.gopath}/src/pk11/bbbb.go
in main.go. imported pk11, when Im use pk11.#, is not support auto complete.
but support json/binary/http and more.

I'm see it's painc at [cfg.Suggest] -> [types.Eval] -> [check.rawExpr] -> [check.exprInternal] -> [check.ident] -> [check.errorf(e.Pos(), "use of package %s not in selector", obj.name)]

And, I'm try install go 1.10.1 and https://github.com/nsf/gocode is OK.
what ???

@RelicOfTesla
Copy link

RelicOfTesla commented Jun 12, 2018

// D:\dev-soft\go\sdk_gopath\src\test\pk1\a.go
package pk1

import (
	"fmt"
	"test/pk2"
)

func main() {
	fmt.Printf("hi")
	pk2.    // <---------
}
// D:\dev-soft\go\sdk_gopath\src\test\pk2\b.go
package pk2

import (
	"fmt"
)

func Hello2() {
	fmt.Printf("hello")
}

test pk2.xxx , character offset is 90

D:\dev-soft\go\sdk_gopath\src\github.com\mdempsky\gocode>gocode -f=json -in=D:\dev-soft\go\sdk_gopath\src\test\pk1\a.go autocomplete 90

> null

Test fmt.xxx, character offset 71


D:\dev-soft\go\sdk_gopath\src\github.com\mdempsky\gocode>gocode -f=json -in=D:\dev-soft\go\sdk_gopath\src\test\pk1\a.go autocomplete 71

> [0,[{"class":"func","package":"fmt","name":"Errorf","type":"func(format string,
> a ...interface{}) error"},{"class":"func","package":"fmt","name":"Fprint","type"
> :"func(w io.Writer, a ...interface{}) (n int, err error)"},{"class":"func","pack
> age":"fmt","name":"Fprintf","type":"func(w io.Writer, format string, a ...interf
> ace{}) (n int, err error)"},{"class":"func","package":"fmt","name":"Fprintln","t
> ype":"func(w io.Writer, a ...interface{}) (n int, err error)"},{"class":"func","
> package":"fmt","name":"Fscan","type":"func(r io.Reader, a ...interface{}) (n int
> , err error)"},{"class":"func","package":"fmt","name":"Fscanf","type":"func(r io
> .Reader, format string, a ...interface{}).........................



D:\dev-soft\go\sdk_gopath\src\github.com\mdempsky\gocode>go version

go version go1.9.5 windows/amd64

D:\dev-soft\go\sdk_gopath\src\github.com\mdempsky\gocode>go env

set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=D:/dev-soft/go/sdk_gopath
set GORACE=
set GOROOT=D:\dev-soft\go
set GOTOOLDIR=D:\dev-soft\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

But, https://github.com/nsf/gocode it's OK!!!!

D:\dev-soft\go\sdk_gopath\src\github.com\nsf\gocode>gocode -f=json -in=D:\dev-soft\go\sdk_gopath\src\test\pk1\a.go autocomplete 90
[0, [{"class": "func", "name": "Hello2", "type": "func()", "package": "test/pk2"
}]]

@ramya-rao-a

@mattn
Copy link

mattn commented Jul 3, 2018

This is known issue. gocode make candidates from current package or standard package. So you must enable source importer with -source flag.

If you use vim-go, please try this.

diff --git a/autoload/go/complete.vim b/autoload/go/complete.vim
index 127d57f..279b10c 100644
--- a/autoload/go/complete.vim
+++ b/autoload/go/complete.vim
@@ -9,6 +9,7 @@ function! s:gocodeCommand(cmd, args) abort
   let cmd = [bin_path]
   let cmd = extend(cmd, ['-sock', socket_type])
   let cmd = extend(cmd, ['-f', 'vim'])
+  let cmd = extend(cmd, ['-source'])
   let cmd = extend(cmd, [a:cmd])
   let cmd = extend(cmd, a:args)
 

@theSuess
Copy link
Author

theSuess commented Jul 3, 2018

I'm using emacs and tried to add the -source flag but still no luck

@mattn
Copy link

mattn commented Jul 3, 2018

Well, -source flags must be specified for client not server.

@ramya-rao-a
Copy link

@mattn And what should be passed in the -source flag? The gopath?

@mattn
Copy link

mattn commented Jul 9, 2018

No extra argument for -source.

@rdner
Copy link

rdner commented Jul 11, 2018

After I added the -source flag to go-autocomplete in emacs autocomplete on my large project got extremely slow, it takes 10-30 seconds sometimes. Since it was completely unusable I had to remove the -source flag back.

Seems like autocomplete for external packages was turned off by default on purpose because of this issue.

@OneOfOne
Copy link

OneOfOne commented Jul 11, 2018

@pragmader You will have to use the cache branch if you want to use -source without the slow down.

Hopefully @mdempsky will merge it soon.

@rdner
Copy link

rdner commented Jul 12, 2018

What about updating the go-autocomplete in melpa to support the -source flag as an option after the cache branch is finished? @mdempsky @nsf

Would be very nice.

@nsf
Copy link

nsf commented Jul 12, 2018

I'm ok with applying that kind of patch to gocode's editor plugins. Send pull requests. Well, in case if upstream package repos are built from it.

@ramya-rao-a
Copy link

@mdempsky I have gotten feedback with completions being slow on using the -source flag in VS Code as well. Any updates on the work being done in the cache branch?

@johanbrandhorst
Copy link

This just started happening for me out of the blue 2 days ago. Internal types work, but as soon as I try to autocomplete for any imported packages there are no candidates. I have tried both with and without -source and there's no difference. Same problem with nsf/gocode as well.

As an example, when developing a personal project (https://github.com/johanbrandhorst/certify):

package certify
import "github.com/cloudflare/cfssl/signer"

import (
        "context"
        "crypto/tls"
        "crypto/x509"
        "encoding/json"
        "net/http"
        "net/url"

        "github.com/cloudflare/cfssl/api/client"
        "github.com/cloudflare/cfssl/auth"
        "github.com/cloudflare/cfssl/csr"
        "github.com/cloudflare/cfssl/signer"
)

// CFSSLIssuer implements the Issuer interface
// with a Cloudflare CFSSL CA server backend.
//
// URL is required.
type CFSSLIssuer struct {
        // URL specifies the URL to the CFSSL server.
        URL *url.URL
        // TLSConfig allows configuration of the TLS config
        // used when connecting to the CFSSL server.
        TLSConfig *tls.Config
        // Profile is the profile on the CFSSL server
        // that should be used. If unset, the default
        // profile will be used.
        Profile string
        // Auth optionally configures the authentication
        // that should be used.
        Auth auth.Provider

        remote        client.Remote
        remoteCertPEM []byte
        csrGenerator  *csr.Generator
}

// Connect creates a new connection to the CFSSL server
// and sends a request to validate server availability. If not called,
// a connection will be made in the first Issue call.
func (m *CFSSLIssuer) Connect(ctx context.Context) error {
        m.remote = client.NewServerTLS(m.URL.String(), m.TLSConfig)
        // Add context to requests
        m.remote.SetReqModifier(func(req *http.Request, _ []byte) {
                *req = *req.WithContext(ctx)
        })
        signer.#

        // Use the Info endpoint as a PING to check server availability
        resp, err := m.remote.Info([]byte(`{}`))
        if err != nil {
                return err
        }

        m.remoteCertPEM = []byte(resp.Certificate)

        m.csrGenerator = &csr.Generator{Validator: func(req *csr.CertificateRequest) error {
                return nil
        }}

        return nil
}

// Issue issues a certificate with the provided options
func (m *CFSSLIssuer) Issue(ctx context.Context, commonName string, conf *CertConfig) (*tls.Certificate, error) {
        if m.remote == nil {
                err := m.Connect(ctx)
                if err != nil {
                        return nil, err
                }
        }

        // Add context to requests
        m.remote.SetReqModifier(func(req *http.Request, _ []byte) {
                *req = *req.WithContext(ctx)
        })

        csrReq := csr.CertificateRequest{
                CN:         commonName,
                KeyRequest: csr.NewBasicKeyRequest(),
        }

        if conf != nil {
                csrReq.Hosts = append(csrReq.Hosts, conf.SubjectAlternativeNames...)
                for _, ip := range conf.IPSubjectAlternativeNames {
                        csrReq.Hosts = append(csrReq.Hosts, ip.String())
                }
        }

        csrPEM, keyPEM, err := m.csrGenerator.ProcessRequest(&csrReq)
        if err != nil {
                return nil, err
        }

        req := signer.SignRequest{
                Request: string(csrPEM),
                Profile: m.Profile,
        }

        reqBytes, err := json.Marshal(&req)
        if err != nil {
                return nil, err
        }

        var certPEM []byte
        if m.Auth != nil {
                certPEM, err = m.remote.AuthSign(reqBytes, nil, m.Auth)
        } else {
                certPEM, err = m.remote.Sign(reqBytes)
        }
        if err != nil {
                return nil, err
        }

        caChainPEM := append(append(certPEM, '\n'), m.remoteCertPEM...)
        tlsCert, err := tls.X509KeyPair(caChainPEM, keyPEM)
        if err != nil {
                return nil, err
        }

        // This can't error since it's called in tls.X509KeyPair above successfully
        tlsCert.Leaf, _ = x509.ParseCertificate(tlsCert.Certificate[0])
        return &tlsCert, nil
}
2018/08/16 10:24:35 -------------------------------------------------------
2018/08/16 10:24:35 Error parsing input file (outer block):
2018/08/16 10:24:35  <GOPATH>/src/github.com/johanbrandhorst/certify/cfssl.go:50:9: expected selector or type assertion, found ';'
2018/08/16 10:24:35  <GOPATH>/src/github.com/johanbrandhorst/certify/cfssl.go:53:2: expected ';', found 'IDENT' resp
2018/08/16 10:24:35 Elapsed duration: 54.732397ms
2018/08/16 10:24:35 Offset: 0
2018/08/16 10:24:35 Number of candidates found: 0
2018/08/16 10:24:35 Candidates are:
2018/08/16 10:24:35 =======================================================

This should include all the different symbols on the signer package imported under github.com/cloudflare/cfssl/signer. Perhaps interestingly, it seems to add the import to the top of the file? This import is not there in the code itself, only in the debug output (file for reference).

I use gocode through vscode so I don't know if that has anything to do with the extra import (@ramya-rao-a ?).

@johanbrandhorst
Copy link

The same is true of variables of imported types:

2018/08/16 10:41:31 Got autocompletion request for '<GOPATH>/src/github.com/johanbrandhorst/certify/cfssl.go'
2018/08/16 10:41:31 Cursor at: 1606
2018/08/16 10:41:31 -------------------------------------------------------
package certify

import (
        "context"
        "crypto/tls"
        "crypto/x509"
        "encoding/json"
        "net/http"
        "net/url"

        "github.com/cloudflare/cfssl/api/client"
        "github.com/cloudflare/cfssl/auth"
        "github.com/cloudflare/cfssl/csr"
        "github.com/cloudflare/cfssl/signer"
)

// CFSSLIssuer implements the Issuer interface
// with a Cloudflare CFSSL CA server backend.
//
// URL is required.
type CFSSLIssuer struct {
        // URL specifies the URL to the CFSSL server.
        URL *url.URL
        // TLSConfig allows configuration of the TLS config
        // used when connecting to the CFSSL server.
        TLSConfig *tls.Config
        // Profile is the profile on the CFSSL server
        // that should be used. If unset, the default
        // profile will be used.
        Profile string
        // Auth optionally configures the authentication
        // that should be used.
        Auth auth.Provider

        remote        client.Remote
        remoteCertPEM []byte
        csrGenerator  *csr.Generator
}

// Connect creates a new connection to the CFSSL server
// and sends a request to validate server availability. If not called,
// a connection will be made in the first Issue call.
func (m *CFSSLIssuer) Connect(ctx context.Context) error {
        m.remote = client.NewServerTLS(m.URL.String(), m.TLSConfig)
        // Add context to requests
        m.remote.SetReqModifier(func(req *http.Request, _ []byte) {
                *req = *req.WithContext(ctx)
        })

        // Use the Info endpoint as a PING to check server availability
        resp, err := m.remote.Info([]byte(`{}`))
        if err != nil {
                return err
        }

        m.remoteCertPEM = []byte(resp.Certificate)

        m.csrGenerator = &csr.Generator{Validator: func(req *csr.CertificateRequest) error {
                req.#
                return nil
        }}

        return nil
}

// Issue issues a certificate with the provided options
func (m *CFSSLIssuer) Issue(ctx context.Context, commonName string, conf *CertConfig) (*tls.Certificate, error) {
        if m.remote == nil {
                err := m.Connect(ctx)
                if err != nil {
                        return nil, err
                }
        }

        // Add context to requests
        m.remote.SetReqModifier(func(req *http.Request, _ []byte) {
                *req = *req.WithContext(ctx)
        })

        csrReq := csr.CertificateRequest{
                CN:         commonName,
                KeyRequest: csr.NewBasicKeyRequest(),
        }

        if conf != nil {
                csrReq.Hosts = append(csrReq.Hosts, conf.SubjectAlternativeNames...)
                for _, ip := range conf.IPSubjectAlternativeNames {
                        csrReq.Hosts = append(csrReq.Hosts, ip.String())
                }
        }

        csrPEM, keyPEM, err := m.csrGenerator.ProcessRequest(&csrReq)
        if err != nil {
                return nil, err
        }

        req := signer.SignRequest{
                Request: string(csrPEM),
                Profile: m.Profile,
        }

        reqBytes, err := json.Marshal(&req)
        if err != nil {
                return nil, err
        }

        var certPEM []byte
        if m.Auth != nil {
                certPEM, err = m.remote.AuthSign(reqBytes, nil, m.Auth)
        } else {
                certPEM, err = m.remote.Sign(reqBytes)
        }
        if err != nil {
                return nil, err
        }

        caChainPEM := append(append(certPEM, '\n'), m.remoteCertPEM...)
        tlsCert, err := tls.X509KeyPair(caChainPEM, keyPEM)
        if err != nil {
                return nil, err
        }

        // This can't error since it's called in tls.X509KeyPair above successfully
        tlsCert.Leaf, _ = x509.ParseCertificate(tlsCert.Certificate[0])
        return &tlsCert, nil
}
2018/08/16 10:41:31 -------------------------------------------------------
2018/08/16 10:41:31 Error parsing input file (outer block):
2018/08/16 10:41:31  <GOPATH>/src/github.com/johanbrandhorst/certify/cfssl.go:59:7: expected selector or type assertion, found ';'
2018/08/16 10:41:31  <GOPATH>/src/github.com/johanbrandhorst/certify/cfssl.go:60:3: expected ';', found 'return'
2018/08/16 10:41:31 Elapsed duration: 73.65926ms
2018/08/16 10:41:31 Offset: 0
2018/08/16 10:41:31 Number of candidates found: 0
2018/08/16 10:41:31 Candidates are:
2018/08/16 10:41:31 =======================================================

Interestingly, the line before this says req is of "invalid type":

2018/08/16 10:41:31 Got autocompletion request for '<GOPATH>/src/github.com/johanbrandhorst/certify/cfssl.go'
2018/08/16 10:41:31 Cursor at: 1603
2018/08/16 10:41:31 -------------------------------------------------------
package certify

import (
        "context"
        "crypto/tls"
        "crypto/x509"
        "encoding/json"
        "net/http"
        "net/url"

        "github.com/cloudflare/cfssl/api/client"
        "github.com/cloudflare/cfssl/auth"
        "github.com/cloudflare/cfssl/csr"
        "github.com/cloudflare/cfssl/signer"
)

// CFSSLIssuer implements the Issuer interface
// with a Cloudflare CFSSL CA server backend.
//
// URL is required.
type CFSSLIssuer struct {
        // URL specifies the URL to the CFSSL server.
        URL *url.URL
        // TLSConfig allows configuration of the TLS config
        // used when connecting to the CFSSL server.
        TLSConfig *tls.Config
        // Profile is the profile on the CFSSL server
        // that should be used. If unset, the default
        // profile will be used.
        Profile string
        // Auth optionally configures the authentication
        // that should be used.
        Auth auth.Provider

        remote        client.Remote
        remoteCertPEM []byte
        csrGenerator  *csr.Generator
}

// Connect creates a new connection to the CFSSL server
// and sends a request to validate server availability. If not called,
// a connection will be made in the first Issue call.
func (m *CFSSLIssuer) Connect(ctx context.Context) error {
        m.remote = client.NewServerTLS(m.URL.String(), m.TLSConfig)
        // Add context to requests
        m.remote.SetReqModifier(func(req *http.Request, _ []byte) {
                *req = *req.WithContext(ctx)
        })

        // Use the Info endpoint as a PING to check server availability
        resp, err := m.remote.Info([]byte(`{}`))
        if err != nil {
                return err
        }

        m.remoteCertPEM = []byte(resp.Certificate)

        m.csrGenerator = &csr.Generator{Validator: func(req *csr.CertificateRequest) error {
                r#
                return nil
        }}

        return nil
}

// Issue issues a certificate with the provided options
func (m *CFSSLIssuer) Issue(ctx context.Context, commonName string, conf *CertConfig) (*tls.Certificate, error) {
        if m.remote == nil {
                err := m.Connect(ctx)
                if err != nil {
                        return nil, err
                }
        }

        // Add context to requests
        m.remote.SetReqModifier(func(req *http.Request, _ []byte) {
                *req = *req.WithContext(ctx)
        })

        csrReq := csr.CertificateRequest{
                CN:         commonName,
                KeyRequest: csr.NewBasicKeyRequest(),
        }

        if conf != nil {
                csrReq.Hosts = append(csrReq.Hosts, conf.SubjectAlternativeNames...)
                for _, ip := range conf.IPSubjectAlternativeNames {
                        csrReq.Hosts = append(csrReq.Hosts, ip.String())
                }
        }

        csrPEM, keyPEM, err := m.csrGenerator.ProcessRequest(&csrReq)
        if err != nil {
                return nil, err
        }

        req := signer.SignRequest{
                Request: string(csrPEM),
                Profile: m.Profile,
        }

        reqBytes, err := json.Marshal(&req)
        if err != nil {
                return nil, err
        }

        var certPEM []byte
        if m.Auth != nil {
                certPEM, err = m.remote.AuthSign(reqBytes, nil, m.Auth)
        } else {
                certPEM, err = m.remote.Sign(reqBytes)
        }
        if err != nil {
                return nil, err
        }

        caChainPEM := append(append(certPEM, '\n'), m.remoteCertPEM...)
        tlsCert, err := tls.X509KeyPair(caChainPEM, keyPEM)
        if err != nil {
                return nil, err
        }

        // This can't error since it's called in tls.X509KeyPair above successfully
        tlsCert.Leaf, _ = x509.ParseCertificate(tlsCert.Certificate[0])
        return &tlsCert, nil
}
2018/08/16 10:41:31 -------------------------------------------------------
2018/08/16 10:41:31 Elapsed duration: 119.334264ms
2018/08/16 10:41:31 Offset: 0
2018/08/16 10:41:31 Number of candidates found: 5
2018/08/16 10:41:31 Candidates are:
2018/08/16 10:41:31   func real(c ComplexType) FloatType
2018/08/16 10:41:31   func recover() interface{}
2018/08/16 10:41:31   type rune rune
2018/08/16 10:41:31   var req *invalid type
2018/08/16 10:41:31   var resp invalid type
2018/08/16 10:41:31 =======================================================

Sounds suspicious I guess?

@ramya-rao-a
Copy link

@mdempsky I believe the source flag is not needed if the dependent packages are already built and installed. Is this a wrong assumption?

@johanbrandhorst
Copy link

I have tracked this down to (in my case) go/internal/gcimporter.Import and subsequently go/internal/gcimporter.FindPkg not finding local packages. I don't understand how, and it's probably due to my environment, but I've added some debugging to the calls that gocode makes to the Importer, and for the path github.com/mdempsky/gocode/internal/lookdot it simply cannot find it (even while operating within the same directoy).

err can't find import: "github.com/mdempsky/gocode/internal/lookdot"`

Using the source importer works, but is unacceptably slow. I've tried deleting $GOPATH/pkg and rebuilding all packages but it still can't find the imports.

@ramya-rao-a is it possible to enable the use of the source flag when making autocomplete requests from VS Code?

@ramya-rao-a
Copy link

@johanbrandhorst It is possible, but like you said it slows things down, so I dont know how helpful that can be. I can have a different build out for the go plugin that uses the source flag, but I wouldnt release it as an offical update until the cache branch is merged

@johanbrandhorst
Copy link

johanbrandhorst commented Aug 26, 2018

@ramya-rao-a I realised I could just rebuild the gocode binary and force source mode, so don't worry :). I will try to keep digging as to why this is happening though. Today I've managed to write a program that successfully calls gcimporter.Import and at the same time have gocode get an error on the same gcimporter.Import call. Very spooky :/.

@kat-co
Copy link

kat-co commented Sep 25, 2018

For emacs friends using company-go from MELPA (which as far as I can tell is pulled from @nsf 's https://github.com/nsf/gocode), you'll need to add this to your emacs initialization to get auto-completion to work:

(setf company-go-gocode-args '("-source"))

Per @nsf 's comment, I will try and find some time to issue a PR against nsf/gocode to make this default for company-go.

Please note that this does incur all the slowdowns that are being discussed here -- even with libraries residing in $GOPATH/pkg. @mdempsky the cache branch will solve this? Any ETA for a merge?

@magodo
Copy link

magodo commented Sep 27, 2018

@mdempsky I believe the source flag is not needed if the dependent packages are already built and installed. Is this a wrong assumption?

I have two go env in one machine, if i build gocode with $GOPATH1, then everything under $GOPATH1 works well without -source(even newly installed pacakge); However, if I switched to $GOPATH2, only standard library is completed.

Can you tell me what's going on here?

@johanbrandhorst
Copy link

So I wiped my entire Arch installation in an attempt to fix this.. and it worked. So it's clearly an environmental problem, though I do not know how or why. If you get desperate (like I did), recreating your entire environment might well work. Just purging all of your Go stuff and reinstalling that also might work, but I was due a reinstall anyway.

@vuleetu
Copy link

vuleetu commented Oct 8, 2018

I had the same issue. After removing old vendor directory, it works now.

@cnbuff410
Copy link

Any updates on this issue? What's the latest status of the cache branch?

@stamblerre
Copy link
Collaborator

I will be investigating merging the cache branch soon.

@stamblerre
Copy link
Collaborator

The caching branch was merged in #71.

@dvcrn
Copy link

dvcrn commented Oct 25, 2018

This is only partly related, but since switching gocode to the mdempsky version with go1.11, all external packages result in invalid type when completing. I tried debugging it myself but didn't get very far. Switching gocode back to the nsf version and it works fine.

-source is added just fine, and even when running gocode in debug with the flag appended, it still suggests invalid type as the type

@stamblerre
Copy link
Collaborator

@dvcrn: do you mind filing a separate issue for your case?

@dvcrn
Copy link

dvcrn commented Oct 26, 2018

Sure, will do!

/e #79

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests