Long file names (length > 260) should have special prefix

[annoynimouse]

...what i hate about java developers - is their love for redundant prefixes in every subfolder.

I thought it might be due to TortoiseGIT, yet the console git did me the same.

I thought my T-Git log would remain in clipboard, silly me.

Okay, i was cloning https://github.com/xwiki/xwiki-platform into c:\XWiki.src\xwiki-platform
It downloaded the pack checked delta's and then started checking out.
It then spitted one error that it cannot create file.
It then continued and finally said it is 100% okay.

To me it seems very strange - at elast one error happened, at least one file was not checked out, yet git reports cloning success. Maybe it is right, dunno, but looks strange.

I looked manually and yes, there was some file and some folder missed,

And git.exe could do northing about that.
See http://pastebin.ca/2172286

The problem is those redundant pathnames are crossing 260 length boundary - see http://jsfiddle.net/eaukk/

To access longer pathnames one should use special prefix and Unicode-aware version of function.
See http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath

[sschuberth]

For reference, we had a discussion about lifting the 260 character limit before on the mailing list:
http://thread.gmane.org/gmane.comp.version-control.msysgit/14572

[the-Arioch]

subscribing my @work account
Thank You for reference.

Archiving pastebin and jsfiddle logs: http://permalink.gmane.org/gmane.comp.version-control.msysgit/16367

However, apart from file length itself - the exit status is strange. I'd re-quote:

It downloaded the pack checked delta's and then started checking out.
 It then spitted one error that it cannot create file.
 It then continued and finally said it is 100% okay.

To me it seems very strange - at least one error happened, at least one file was not checked out, yet git reports cloning success. Maybe it is right, dunno, but looks strange.

As far as got it, on UNIX git has 4KB length limit.
Maybe some could create 3900-length local path in Linux and try to clone some repositories into it ?
Would git/UNIX act the same, failing to create files, yet claiming cloning was 100% successfull ?

[kblees]

Git only supports working directories up to 1024 characters (see abspath.c:27). This is probably for good reason, I don't know (so that hitting the 4k boundary on Unix is impossible?).

Anyway, I can confirm that 'git clone' on Windows returns success even if some files could not be checked out correctly. This is probably worth fixing.

However, supporting long path names in general is pointless as long as Windows shells and applications don't support this. Quote from the MSDN article linked above: "The shell and the file system have different requirements. It is possible to create a path with the Windows API that the shell user interface is not able to interpret properly."

See also this discussion: http://thread.gmane.org/gmane.comp.version-control.msysgit/16284

[the-Arioch]

The MSDN quotation is more about names like "12345 " or " 12345"
I recall some of that caused problems in GUI.

I also have doubts about case-sensitive names. Whle the core supports it, i guess both GUi and CLI does not.

---

Isn't it too early to declare somethign useless if some programs do not support it ?

If developer needs it, he would find better program or push for bug fixing. But even if some obstaclle would be fatal to him - let it be obstacle in another program. As of now the showstopper is Git. Git can be pointed at and claimed obsolete. Not some other program. And Git can harder get replacement than filemanager.

[kusma]

I don't think anyone actually declared it as useless, only warned about possible issues along the road if someone dares to go down this path.

This is one of those issues you don't simply request and magically get granted, though. Someone who cares about it needs to pick it up and work on it. Luckily for whoever does (if any), Karsten's work on unicode pathnames already did a lot of the heavy lifting needed.

[the-Arioch]

Okay, cmd.exe does not work with it, so no cmd/bat scripts.
PowerShell fails too.

I tried Notepad++ and it crashed, made bugreport.
I launched empty N++ and did File|Open and... it was clever trick. Windows or N++ dunno, but they suddenly converted name to short DOS 8.3 names. So, converting path to 8.3 and leaving only file itself long might be a clever trick even for older applications :-)

Java.exe finally read such files, hopefully any Java program as well

[kblees]

Have you already tried paths that exceed the 260 limit even with 8.3 names?

The problem is not Notepad++ (or any other editor / IDE, for that matter), it is Windows Explorer and the shell's common file open dialog that cannot handle long names. That means that practically all Windows programs that work with files are affected.

That beeing said, there are a few workarounds that you might try:

• I believe Cygwin can handle long names, so you could use cygwin's file managers, editors / IDEs and of course git
• Subst your repository to a new drive letter to shorten the paths (i.e. "subst D: c:\XWiki.src\xwiki-platform", that gives you 24 more characters to work with...)
• Restructure the repository to use shorter paths.

[nitram509]

+1 for this issue.

It seems, that the underlying "old" cygwin has some influence.

My test setup is Windows7 64bit "Git version 1.7.8-preview20111206".
I've created some source folders with a length of 260+ long path names.
"git add" shows the following error:
fatal: unable to stat 'Adobe InDesign CS6 Server SDK/tools/benchmark/BenchmarkChartsConsole/src/com/adobe/ids/benchmark/
samples/dce/source/suribachi/src/suribachi/api/vo/action/tools/benchmark/BenchmarkChartsConsole/src/BenchmarkChartsConso
le-app.xml': No such file or directory
Of course, this file exists but has a long path.

Additionally I've throwed in this command:
"find . | wc -L"
Pops up this error (and much more):
find: ./Adobe InDesign CS6 Server SDK/tools/benchmark/BenchmarkChartsConsole/src/com/adobe/ids/benchmark/samples/samplec
lient-java-soap/src/com/adobe/ids/samples/sampleclient-aspnet-soap/sampleclient-aspnet-vb-soap/Default.aspx.vb: File or
path name too long

Than I've checked the find command within cygwin (CYGWIN_NT-6.1-WOW64 momo 1.7.15(0.260/5/3) 2012-05-09 10:25 i686 Cygwin)
Again "find . | wc -L" ... no errors,
result : 299

So maybe just linking git against a newer cygwin should work?

[the-Arioch]

@kblees Have you already tried paths that exceed the 260 limit even with 8.3 names?

I told you, "I launched empty N++ and did File|Open and... it was clever trick"
It was that very standard FileOpen dialog of Windows (of Windows 7 x64). Maybe earlier Windows would fail and commandline would surely failand many standalone apps might, yet Windows GUI itself is ready for it!

Also both N++ and Delphi XE2 IDe mananged to read the file by 8.3 names
And why would not ? 8.3 names are not shortcuts but valid names for Win32 subsystem. And if filesystem driver then convert it, the drivers level are not affected by MAX_PATH bottleneck.

Also command-line java.exe (as of JRE 7b05 x64) does access those files. What interesting is Java approach at dir traversal: log shows it opens each time from root to a subfolder, then to sub-sub-folder and so on, one by one. Hopefully that allows them to alleviate lack of relating paths in extended-length strings.

[the-Arioch]

I believe Cygwin can handle long names,

If having no other choice and hopefully support from Tortoise, would try.
Java proven working so hopefully that makes sense.

Also, since Java works, i might be able to get this with Eclipse's Jit. Attempt at cloning failed, but maybe attempt to check-out would succeed.

Restructure the repository to use shorter paths.

Not the option i can make on 3rd-party repo.
And i don't know a bit of java and its build/deploy infrastructure and conventions. I just hope it "just works" so i can have a chance at "hit-and-run" patching.

---

Subst your repository to a new drive letter

Dunno how it is implemented. In DOS it was just a redirector after user code before actual File IO API.
If that would be the same, it would change nothing.

Well, it really seems to work even wit hrelative paths.

---

C:\Users\burov>cd \fs160.src\xwiki-platform-core\xwiki-platform-classloader\xwik
i-platform-classloader-protocols\xwiki-platform-classloader-protocol-attachmentj
ar\src\main\java

C:\fs160.src\xwiki-platform-core\xwiki-platform-classloader\xwiki-platform-class
loader-protocols\xwiki-platform-classloader-protocol-attachmentjar\src\main\java

subst L: .

C:\fs160.src\xwiki-platform-core\xwiki-platform-classloader\xwiki-platform-class
loader-protocols\xwiki-platform-classloader-protocol-attachmentjar\src\main\java

subst
L:: => C:\fs160.src\xwiki-platform-core\xwiki-platform-classloader\xwiki-platfo
rm-classloader-protocols\xwiki-platform-classloader-protocol-attachmentjar\src\m
ain\java

C:\fs160.src\xwiki-platform-core\xwiki-platform-classloader\xwiki-platform-class
loader-protocols\xwiki-platform-classloader-protocol-attachmentjar\src\main\java

l:

L:>"c:\Notepad++\notepad++.exe" org\xwiki\classloader\internal\protocol\attachm
entjar\AttachmentURLStreamHandler.java\need13more\over260now\DLL_Load_Test.dpr

L:>dir org\xwiki\classloader\internal\protocol\attachmentjar\AttachmentURLStrea
mHandler.java\need13more\over260now\DLL_Load_Test.dpr
Том в устройстве L не имеет метки.
Серийный номер тома: B2AB-1E7A

Содержимое папки L:\org\xwiki\classloader\internal\protocol\attachmentjar\Attac
hmentURLStreamHandler.java\need13more\over260now

06.07.2012 15:47 2 482 DLL_Load_Test.dpr
1 файлов 2 482 байт
0 папок 164 744 040 448 байт свободно

L:>type org\xwiki\classloader\internal\protocol\attachmentjar\AttachmentURLStre
amHandler.java\need13more\over260now\DLL_Load_Test.dpr
program DLL_Load_Test;

uses
Vcl.Forms,

..........

[the-Arioch]

Anyway, I can confirm that 'git clone' on Windows returns success even if some files could not be checked out correctly. > This is probably worth fixing.

Was it fixed ? or should extra ticket be created to make it not forgotten ?

[kusma]

Is it a bug specific to Git for Windows? If so, are you experiencing issue #43?

[the-Arioch]

don't know, at least it was reproduced there above

404 #43

[kusma]

msysgit/msysgit#43

[the-Arioch]

yes, happens in my Git-1.7.11-preview20120710.exe

[sschuberth]

@the-Arioch kusma's question was whether you see "git clone" to report success although it fails when using the git wrapper from the "cmd" directory, or when using the native git executable.

[the-Arioch]

this i understand.

[sschuberth]

So why don't you answer his question then? In any case, if you still see "git clone" not fail when it should, please file a new issue instead of hooking this one.

[the-Arioch]

sorry ? between your msg and his link there is what exactly if not my answer ?

[sschuberth]

You are not saying what git executable you are using, the one from "<installation path>\bin\git.exe" or "<installation path>\cmd\git.exe". I believe you are using the first one, which is why your issue is not related to issue 43.

[the-Arioch]

i would check when be at that box. I wonder if i can run directly from BIN - there are some pre-requisites for environment that wrappers did establish (git.cmd is rather complex script).

I'm pretty sure that Tortoise used bin\git while from command line i used cmd\git

[nitram509]

It seems to me, that some issues are mixed here.
I've created a separate one: msysgit/msysgit#52

[kblees]

Anyway, I can confirm that 'git clone' on Windows returns success even if some files could not be checked out correctly. This is probably worth fixing.

Was it fixed ? or should extra ticket be created to make it not forgotten ?

Not that I know of. It turns out that 'git clone' simply doesn't check the result of the checkout operation, and never has (see unpack_trees in builtin/clone.c:579). This is not a Git for Windows problem, so please discuss it on the git mailing list.

[schwaerz]

I'm glad to see that there's still some discussion about this topic.
We're facing issues here because we've a test environment using cygwin which unfortunately allows paths to be created beyond that 260 character border.
Next what happens is that 'git clean -fdx' on our jenkins fails because it just can't delete these files. However we don't like to use the cygwin git build because it's not the official windows git implementation.

I personally did invest some effort on fixing this issue (focusing only on getting 'git clean' to work) but that would rather be a change in the msys support layer for which I didn't find yet some good place where to put my automated tests... Question is whether these changes somehow would need to be done in msys/mingw and than merged into git for windows? And where to put any tests?

At this moment I'm unsure whether it's really worth the effort - we could just stick to the cygwin implementation of git - while it doesn't look too easy to do changes in msys/mingw.

[sschuberth]

This issue by now contains various comments that are not related to the original issue, so I'm closing this in favor of msysgit/msysgit#52 as the long path issue is a Windows / MSYS one, not a core Git one.

[the-Arioch]

the mentioned link claims "However, git.exe is not an MSYS binary, and does not link to the MSYS-runtime."

[sschuberth]

Which is perfectly true as you can easily verify with a tool like Dependency Walker.

[the-Arioch]

No, DepWalk can detect statically bound dynamically loaded RTLs, but one needs different tools to detect statically compiled RTL (lib, obj) or dynamically bound one (Win32 LoadLibrary call). However that is just nitpicking.

The point is that since GIT is not MSYS application, it looks very strange to close this bug just because some other unrelated program (MSYS) also has one. MSYS is MSYS and GIT is GIT.

[sschuberth]

If git.exe was an MSYS binary it would link against msys-1.0.dll dynamically, which can be checked with Dependency Walker. No need to play the wise guy by mentioning irrelevant cases where Dependency Walker would not work. That's just not helpful.

Also, I think there is no need to lecture us what MSYS is and what not. We got the point that some users would like to see support for long paths, and in order to do so we would need to fix both git.exe and the MSYS tools it relies on. IMHO, a single ticket is enough to track this feature request, and I've chosen msysgit/msysgit#52 instead of this one as it has a clearer comment history and is not spammed by nitpicking / irrelevant comments.