Patchwork [1,of,6,v2] util: add platform-agnostic absjoin function and some long-paths win utils

login
register
mail settings
Submitter Kostia Balytskyi
Date Sept. 19, 2017, 4:57 p.m.
Message ID <280ac0656c0637d1006b.1505840265@devvm1416.lla2.facebook.com>
Download mbox | patch
Permalink /patch/24024/
State Deferred, archived
Headers show

Comments

Kostia Balytskyi - Sept. 19, 2017, 4:57 p.m.
# HG changeset patch
# User Kostia Balytskyi <ikostia@fb.com>
# Date 1505835722 25200
#      Tue Sep 19 08:42:02 2017 -0700
# Node ID 280ac0656c0637d1006b956cfe4b21808ff0207c
# Parent  2844c4bd5a398aaa78114d02f6a6a876898cb910
util: add platform-agnostic absjoin function and some long-paths win utils

The goal of this seies is to add support for long paths on Windows.
Context here is that Win32 API CreateFileA (which opens a file handle and
which is what CreateFile macro is resolved to by default) does not recognize
paths longer than MAX_PATH constant (260 chars), unless the path starts
with \\?\, which causes API to not do any checks or preprocessing of the
file path. Therefore, it is posible to use long paths with this prefix,
but it comes at a cost: forward slashes are not automatically replaced by
baclslashes, . and .. are not automatically resolved to appropriate dirs.

Also, it is hard to change paths to use //?/ everywhere, so I propose that
the approach is to handle both types of paths simultaneously.

Also, even if this series is not accepted as it is, I would like to start
a discussion about solving this problem in general.

This particular patch adds some first steps in reaching this goal:
- it adds helper functions hasntprefix and converttontpath
- it adds absjoin function which is supposed to replace _join method
of dirstate

Patch

diff --git a/mercurial/posix.py b/mercurial/posix.py
--- a/mercurial/posix.py
+++ b/mercurial/posix.py
@@ -673,3 +673,9 @@  def bindunixsocket(sock, path):
     if bakwdfd:
         os.fchdir(bakwdfd)
         os.close(bakwdfd)
+
+def absjoin(absprefix, relpath):
+    """Join a relative path to an absolute path. This function assumes that
+    the passed absolute path already contains a terminating directory
+    separator."""
+    return absprefix + relpath
diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -150,6 +150,7 @@  testpid = platform.testpid
 umask = platform.umask
 unlink = platform.unlink
 username = platform.username
+absjoin = platform.absjoin
 
 try:
     recvfds = osutil.recvfds
diff --git a/mercurial/windows.py b/mercurial/windows.py
--- a/mercurial/windows.py
+++ b/mercurial/windows.py
@@ -245,6 +245,28 @@  def normpath(path):
 def normcase(path):
     return encoding.upper(path) # NTFS compares via upper()
 
+def converttontpath(path):
+    """Prepend \\?\ prefix to a path and perform slash replacements
+    The '\\?\' prefix tells Win32 API to not perform userland path checks,
+    therefore allowing long paths. This also means that / are not replaced
+    with \ by Win32, so we need to do it manually. See MSDN entry on
+    CreateFile function for more details"""
+    return '\\\\?\\' + os.path.normpath(path)
+
+def hasntprefix(path):
+    """Check if path starts with a \\?\ prefix"""
+    return path.startswith('\\\\?\\')
+
+def absjoin(absprefix, relpath):
+    """See docblock for posix.absjoin for explanation of what this does."""
+    if hasntprefix(absprefix):
+        # we assume that if absprefix starts with \\?\, this means that it
+        # was already preprocessed by converttontpath and therefore does
+        # not need to passed to `localpath`
+        return absprefix + localpath(relpath)
+    else:
+        return converttontpath(absprefix + relpath)
+
 # see posix.py for definitions
 normcasespec = encoding.normcasespecs.upper
 normcasefallback = encoding.upperfallback