Patchwork D851: [RFC] setup: increase MAX_PATH limit on Windows 10 Anniversary Update

login
register
mail settings
Submitter phabricator
Date Sept. 30, 2017, 9:46 a.m.
Message ID <differential-rev-PHID-DREV-rmkoj4vkhujcquiuv66p-req@phab.mercurial-scm.org>
Download mbox | patch
Permalink /patch/24233/
State RFC
Headers show

Comments

phabricator - Sept. 30, 2017, 9:46 a.m.
indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As documented at
  https://msdn.microsoft.com/en-gb/library/windows/desktop/aa365247(v=vs.85).aspx
  system settings or per-application settings can now opt in to increasing
  the length of MAX_PATH, removing the 260 character restriction. This
  impacts the following API calls:
  
  CreateDirectoryW, CreateDirectoryExW, GetCurrentDirectoryW,
  RemoveDirectoryW, SetCurrentDirectoryW, CopyFileW, CopyFile2,
  CopyFileExW, CreateFileW, CreateFile2, CreateHardLinkW,
  CreateSymbolicLinkW, DeleteFileW, FindFirstFileW, FindFirstFileExW,
  FindNextFileW, GetFileAttributesW, GetFileAttributesExW,
  SetFileAttributesW, GetFullPathNameW, GetLongPathNameW, MoveFileW,
  MoveFileExW, MoveFileWithProgressW, ReplaceFileW, SearchPathW,
  FindFirstFileNameW, FindNextFileNameW, FindFirstStreamW,
  FindNextStreamW, GetCompressedFileSizeW, GetFinalPathNameByHandleW.
  
  Adding an application manifest to hg.exe will activate this setting
  and possibly eliminate some 260 character path length limitations
  in Mercurial without having to convert all paths to absolute and use
  the "\\?\" prefix, which was previously the only supported mechanism
  for using longer paths.
  
  I HAVE NOT YET TESTED THIS PATCH.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D851

AFFECTED FILES
  setup.py

CHANGE DETAILS




To: indygreg, #hg-reviewers
Cc: mercurial-devel
phabricator - Sept. 30, 2017, 11:27 a.m.
abuehl added a comment.


  What about file I/O done via Python lib (e.g. python27.dll)?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D851

To: indygreg, #hg-reviewers
Cc: abuehl, mercurial-devel
phabricator - Sept. 30, 2017, 11:34 a.m.
indygreg added a comment.


  In https://phab.mercurial-scm.org/D851#14300, @abuehl wrote:
  
  > What about file I/O done via Python lib (e.g. python27.dll)?
  
  
  I /think/ this works at the process - not DLL - level. But I need to test on a Windows machine to be sure. One of the reasons this is marked RFC.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D851

To: indygreg, #hg-reviewers
Cc: abuehl, mercurial-devel
phabricator - Sept. 30, 2017, 11:53 a.m.
abuehl added a comment.


  In https://phab.mercurial-scm.org/D851#14301, @indygreg wrote:
  
  > In https://phab.mercurial-scm.org/D851#14300, @abuehl wrote:
  >
  > > What about file I/O done via Python lib (e.g. python27.dll)?
  >
  >
  > I /think/ this works at the process - not DLL - level. But I need to test on a Windows machine to be sure. One of the reasons this is marked RFC.
  
  
  Maybe, but what does the Python library itself do with long paths?
  What do you expect from calling e.g. os.unlink with a long path? Can the Python lib handle it?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D851

To: indygreg, #hg-reviewers
Cc: abuehl, mercurial-devel
phabricator - Sept. 30, 2017, 12:52 p.m.
abuehl added a comment.


  In addition, Mercurial is not using the ...W Windows API functions.
  
  I think, for example
  
    https://phab.mercurial-scm.org/diffusion/HG/browse/default/mercurial/cext/osutil.c;da8bdeb1be28b976909a963c89e974264686e2bb$1207
  
  would have to be changed to call CreateFileW in order for this to have an effect.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D851

To: indygreg, #hg-reviewers
Cc: abuehl, mercurial-devel
phabricator - Dec. 24, 2017, 5:54 p.m.
indygreg abandoned this revision.
indygreg added a comment.


  Support for this was added in https://phab.mercurial-scm.org/rHGed5acd3fd7e19b92033b379580c228d88c09785f.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D851

To: indygreg, #hg-reviewers
Cc: abuehl, mercurial-devel

Patch

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -514,6 +514,16 @@ 
             f.write('docs = ')
             f.write(out)
 
+# ws2:longPathAware increases MAX_PATH on Windows 10 Anniversary Edition and
+# newer to larger than its default of 260.
+HGEXE_MANIFEST = '''
+<application xmlns="urn:schemas-microsoft-com:asm.v3">
+    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
+        <ws2:longPathAware>true</ws2:longPathAware>
+    </windowsSettings>
+</application>
+'''.strip()
+
 class buildhgexe(build_ext):
     description = 'compile hg.exe from mercurial/exewrapper.c'
 
@@ -556,11 +566,20 @@ 
             f.write('#define HGPYTHONLIB "%s"\n' % pythonlib)
         objects = self.compiler.compile(['mercurial/exewrapper.c'],
                                          output_dir=self.build_temp)
-        dir = os.path.dirname(self.get_ext_fullpath('dummy'))
-        target = os.path.join(dir, 'hg')
-        self.compiler.link_executable(objects, target,
-                                      libraries=[],
-                                      output_dir=self.build_temp)
+
+        try:
+            fd, manifestpath = tempfile.mkstemp('.manifest')
+            fd.write(HGEXE_MANIFEST)
+            fd.close()
+
+            dir = os.path.dirname(self.get_ext_fullpath('dummy'))
+            target = os.path.join(dir, 'hg')
+
+            self.compiler.link_executable(objects, target,
+                libraries=[], output_dir=self.build_temp,
+                extra_postargs=['/MANIFESTFILE:%s' % manifestpath])
+        finally:
+            os.unlink(manifestpath)
 
     @property
     def hgexepath(self):