Pkg.clone shows password in cleartext on screen when cloning via https

[davidanthoff]

When I Pkg.clone a private repo on github using the https URL, I get a prompt that asks for the username, and then a prompt that asks for the password. When I type my password, I can see it in cleartext on the screen. Instead, * should be shown on the screen for each character I type.

I think this should get the labels bug and assigned to the 0.5.0 milestone because this is quite a no-no from a security point of view.

@tkelman (Tony, feel free to stop me from CCing you on the things that I think should be assigned to 0.5.0, but I'm treating you as the release manager until I hear push back from you :)

julia> versioninfo()
Julia Version 0.5.0-dev+5435
Commit 894fbe7* (2016-07-14 16:00 UTC)
Platform Info:
  System: NT (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

[tkelman]

It was either @wildart or @stevengj who wrote the authentication code I believe.

Re: pinging me personally, I'd rather you didn't, x-ref #15068 (comment)

[tkelman]

I can reproduce this (on Windows, not on Linux). @wildart what's the best way to find out where in the call chain libgit2 gets to for cloning a private https package? wrong issue for that question

[tkelman]

I'm very confused. When I call LibGit2.prompt with password=true by itself, it works as it should. But not within Pkg.clone which should be calling

julia/base/libgit2/callbacks.jl

Line 175 in edb112a

userpass = prompt("Password for '$schema$username@$host'", password=true)

julia> a = LibGit2.prompt("foo", password=true);
foo:
julia> a
"blah"

[wildart]

ha, it's a terminal issue. Base.getpass is a culprit.

[tkelman]

Happy to test if you have a proposed patch/PR.

[stevengj]

For getpass on Windows, see #8228. We ended up using the implementation based on _getch, which is supposed to read a character without echoing it on the terminal.

[wildart]

I can confirm that Base.getpass on windows doesn't mask input and hangs causing #17423

[tkelman]

It looks like it works differently depending on whether you're running from mintty (in cygwin or msys2) or cmd.

[wildart]

I tried 0.4.5 and ccall(:_getch, UInt8, ()) works fine. On master cygwin build in mintty.exe this call hangs.

[tkelman]

It's not a difference between 0.4 and 0.5, it's a difference in what terminal you're using to run Julia. _getch doesn't know how to deal with the named pipes that cygwin's mintty terminal emulator uses. We might need to use something smarter if we want this to work properly in mintty. But note that this is a separate issue from the libgit2 authentication problem, since that happens even outside of mintty. As I said above, LibGit2.prompt and Base.getpass work fine from cmd, but Pkg.clone of a private package doesn't.

[wildart]

Question is from where Pkg.clone was executed? If from mintty terminal then it would hang because of faulty _getch.

[tkelman]

Question is from where Pkg.clone was executed?

cmd in my tests. Probably powershell for @davidanthoff, but that's usually the same console host.

[wildart]

I'm running julia from cmd and it works fine, powershell as well. I only have problems with _getch call on mintty.

@davidanthoff What terminal do you use when running julia?

[tkelman]

You can clone a private package over https with recent 0.5-dev in cmd, without the password getting echoed? I seem to get the password echoed sometimes in cmd if Pkg.clone is the first thing I run, but not if I do anything else in the REPL first. Something might be racy or very sensitive to REPL state.

[wildart]

Yes, no echo for me. The only problem that enter key is swallowed and following output starts right after password prompt.

[davidanthoff]

I'm not using mintty, I'm just starting julia from the startmenu shortcut to reproduce this, so there is also no powershell involved.

Things got even weirder just now. I essentially got this output (I added line numbers in [] that are not present in the original output, and replaced my password with mypassword):

   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0-dev+5478 (2016-07-18 02:34 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 43f3c05 (0 days old master)
|__/                   |  x86_64-w64-mingw32

julia> Pkg.clone("https://github.com/davidanthoff/Judyp.jl.git")
[1] INFO: Cloning Judyp from https://github.com/davidanthoff/Judyp.jl.git
[2] Username for 'https://github.com':davidanthoff
[3] Password for 'https://davidanthoff@github.com':mypassword
[4] Username for 'https://github.com':Password for 'https://mypassword@github.com':

I typed in everything. In line 2, I entered my username and then hit Enter. In line 3 I entered my password and hit Enter. Then nothing happened. Then I hit Enter again, and line 4 appeared. And that is now echoing back the password that I entered as the username in front of the @github thing.

[davidanthoff]

Oh, and I get the same behavior that @tkelman mentioned, i.e. if I execute any simple julia command at the REPL first, things work:

   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0-dev+5478 (2016-07-18 02:34 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 43f3c05 (0 days old master)
|__/                   |  x86_64-w64-mingw32

julia> 5*5
25

julia> Pkg.clone("https://github.com/davidanthoff/Judyp.jl.git")
INFO: Cloning Judyp from https://github.com/davidanthoff/Judyp.jl.git
Username for 'https://github.com':davidanthoff
Password for 'https://davidanthoff@github.com':INFO: Computing changes...
INFO: No packages to install, update or remove

julia>

There should still be a line-break before the INFO: Computing changes..., but no password is echoed.

[stevengj]

Note that Python's getpass function similarly calls msvcrt.getwch on Windows (see win_getpass), which calls _getwch() (see msvcrt_getwch). Does Python's getpass function work in mintty?

[stevengj]

cc @Keno on the terminal weirdness here.

[tkelman]

No the native version of Python is pretty much unusable in mintty, which we've generally been doing a much better job of. How hard can it be to not echo input back to the screen, do we need to do a ccall for that?

[stevengj]

Most of the sources I can find suggest that you use tcgetattr/tcsetattr suppress terminal output (see e.g. here). No idea how that interacts with mintty. (My fear is that getch_ does this already, so we won't gain anything.)

[stevengj]

We can look at how pdcurses and cygwin's ncurses work.

[Keno]

Maybe just pop up a text box? Windows does have that available as an API function if I remember correctly.

[stevengj]

(See also mintty/mintty#56)

[tkelman]

that might be better if we're prompting for interaction anyway

[Keno]

Unfortunately I can't find the API function with text input. There's one with buttons, but unless we want the user to encode their password in ternary first that probably doesn't work.

[vtjnash]

How hard can it be to not echo input back to the screen, do we need to do a ccall for that?

extremely awful. We fork stty.exe twice every time you hit a character at the repl to provide this. (the alternative is to build against cygwin1.dll and hope the user gets lucky with libc dll versioning + fork + rebase triple hell all matching against the buildbot)

If GPL2 is OK for this, there's pinentry, https://www.gnupg.org/download/, implementing the Assuan protocol https://www.gnupg.org/documentation/manuals/assuan/index.html

[Keno]

Found something: https://msdn.microsoft.com/en-us/library/windows/desktop/aa375177(v=vs.85).aspx

[tkelman]

I don't think GPL is okay for this unless we make a separate wrapper program that we shell out to. winapi looks more promising.

[Keno]

A little antiquated perhaps, but it's ok:

[tkelman]

That's probably why it says to use the newer API when you're on Vista or newer?

[Keno]

Yes, the new API is super complicated, but I managed to figure it out as well just now (though this is the server version, so the UI is different from what it would be on the desktop version):

[stevengj]

Some software (e.g. winpty) actually just creates a hidden console window for user input.

[davidanthoff]

I think any of these dialog boxes would be ok for now, i.e. fix this security bug. Long term it would be nicer if the password prompt was console based, but imo that could come after 0.5.0.

[davidanthoff]

I'm closing this, seems resolved.

[stevengj]

Yes, fixed by #17506.