Patchwork [5,of,7] py3: fix manifestdict.fastdelta() to be compatible with memoryview

login
register
mail settings
Submitter Yuya Nishihara
Date March 26, 2017, 11:59 a.m.
Message ID <5e8e48cb4f83a583515b.1490529588@mimosa>
Download mbox | patch
Permalink /patch/19682/
State Accepted
Headers show

Comments

Yuya Nishihara - March 26, 2017, 11:59 a.m.
# HG changeset patch
# User Yuya Nishihara <yuya@tcha.org>
# Date 1490522808 -32400
#      Sun Mar 26 19:06:48 2017 +0900
# Node ID 5e8e48cb4f83a583515b4caaf97045b86747c499
# Parent  435eb9e60147c1cfac43b204b637b8ac8d4830c0
py3: fix manifestdict.fastdelta() to be compatible with memoryview

This doesn't look nice, but a straightforward way to support Python 3.
bytes(m[start:end]) is needed because a memoryview doesn't support ordering
operations. On Python 2, m[start:end] returns a bytes object even if m is
a buffer, so calling bytes() should involve no additional copy.

I'm tired of trying cleaner alternatives, including:

 a. extend memoryview to be compatible with buffer type
    => memoryview is not an acceptable base type
 b. wrap memoryview by buffer-like class
    => zlib complains it isn't bytes-like
Augie Fackler - March 27, 2017, 1:57 p.m.
On Sun, Mar 26, 2017 at 08:59:48PM +0900, Yuya Nishihara wrote:
> # HG changeset patch
> # User Yuya Nishihara <yuya@tcha.org>
> # Date 1490522808 -32400
> #      Sun Mar 26 19:06:48 2017 +0900
> # Node ID 5e8e48cb4f83a583515b4caaf97045b86747c499
> # Parent  435eb9e60147c1cfac43b204b637b8ac8d4830c0
> py3: fix manifestdict.fastdelta() to be compatible with memoryview

Queued these last two, many thanks.

>
> This doesn't look nice, but a straightforward way to support Python 3.
> bytes(m[start:end]) is needed because a memoryview doesn't support ordering
> operations. On Python 2, m[start:end] returns a bytes object even if m is
> a buffer, so calling bytes() should involve no additional copy.
>
> I'm tired of trying cleaner alternatives, including:
>
>  a. extend memoryview to be compatible with buffer type
>     => memoryview is not an acceptable base type
>  b. wrap memoryview by buffer-like class
>     => zlib complains it isn't bytes-like
>
> diff --git a/mercurial/manifest.py b/mercurial/manifest.py
> --- a/mercurial/manifest.py
> +++ b/mercurial/manifest.py
> @@ -650,10 +650,10 @@ def _msearch(m, s, lo=0, hi=None):
>      that string.  If start == end the string was not found and
>      they indicate the proper sorted insertion point.
>
> -    m should be a buffer or a string
> -    s is a string'''
> +    m should be a buffer, a memoryview or a byte string.
> +    s is a byte string'''
>      def advance(i, c):
> -        while i < lenm and m[i] != c:
> +        while i < lenm and m[i:i + 1] != c:
>              i += 1
>          return i
>      if not s:
> @@ -664,10 +664,10 @@ def _msearch(m, s, lo=0, hi=None):
>      while lo < hi:
>          mid = (lo + hi) // 2
>          start = mid
> -        while start > 0 and m[start - 1] != '\n':
> +        while start > 0 and m[start - 1:start] != '\n':
>              start -= 1
>          end = advance(start, '\0')
> -        if m[start:end] < s:
> +        if bytes(m[start:end]) < s:
>              # we know that after the null there are 40 bytes of sha1
>              # this translates to the bisect lo = mid + 1
>              lo = advance(end + 40, '\n') + 1
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Patch

diff --git a/mercurial/manifest.py b/mercurial/manifest.py
--- a/mercurial/manifest.py
+++ b/mercurial/manifest.py
@@ -650,10 +650,10 @@  def _msearch(m, s, lo=0, hi=None):
     that string.  If start == end the string was not found and
     they indicate the proper sorted insertion point.
 
-    m should be a buffer or a string
-    s is a string'''
+    m should be a buffer, a memoryview or a byte string.
+    s is a byte string'''
     def advance(i, c):
-        while i < lenm and m[i] != c:
+        while i < lenm and m[i:i + 1] != c:
             i += 1
         return i
     if not s:
@@ -664,10 +664,10 @@  def _msearch(m, s, lo=0, hi=None):
     while lo < hi:
         mid = (lo + hi) // 2
         start = mid
-        while start > 0 and m[start - 1] != '\n':
+        while start > 0 and m[start - 1:start] != '\n':
             start -= 1
         end = advance(start, '\0')
-        if m[start:end] < s:
+        if bytes(m[start:end]) < s:
             # we know that after the null there are 40 bytes of sha1
             # this translates to the bisect lo = mid + 1
             lo = advance(end + 40, '\n') + 1