Patchwork [09,of,11] py3: do not pass a memoryview to bdiff.bdiff()

login
register
mail settings
Submitter Yuya Nishihara
Date March 3, 2018, 1:27 p.m.
Message ID <5e6c27a228b656f730cf.1520083662@mimosa>
Download mbox | patch
Permalink /patch/28774/
State Accepted
Headers show

Comments

Yuya Nishihara - March 3, 2018, 1:27 p.m.
# HG changeset patch
# User Yuya Nishihara <yuya@tcha.org>
# Date 1520079865 18000
#      Sat Mar 03 07:24:25 2018 -0500
# Node ID 5e6c27a228b656f730cf0afd70e864ec96fc29c2
# Parent  d02adc818848486b23894e5fea4f632fb56b5aa4
py3: do not pass a memoryview to bdiff.bdiff()

This doesn't look nice, but I don't know how to make a zero-copy slice of
bytes which is compatible with the buffer protocol.
Gregory Szorc - March 3, 2018, 3:23 p.m.
On Sat, Mar 3, 2018 at 8:27 AM, Yuya Nishihara <yuya@tcha.org> wrote:

> # HG changeset patch
> # User Yuya Nishihara <yuya@tcha.org>
> # Date 1520079865 18000
> #      Sat Mar 03 07:24:25 2018 -0500
> # Node ID 5e6c27a228b656f730cf0afd70e864ec96fc29c2
> # Parent  d02adc818848486b23894e5fea4f632fb56b5aa4
> py3: do not pass a memoryview to bdiff.bdiff()
>
> This doesn't look nice, but I don't know how to make a zero-copy slice of
> bytes which is compatible with the buffer protocol.
>
> diff --git a/mercurial/mdiff.py b/mercurial/mdiff.py
> --- a/mercurial/mdiff.py
> +++ b/mercurial/mdiff.py
> @@ -30,9 +30,17 @@ blocks = bdiff.blocks
>  fixws = bdiff.fixws
>  patches = mpatch.patches
>  patchedsize = mpatch.patchedsize
> -textdiff = bdiff.bdiff
> +_textdiff = bdiff.bdiff
>  splitnewlines = bdiff.splitnewlines
>
> +# On Python 3, util.buffer() creates a memoryview, which appears not
> +# supporting the buffer protocol
>

memoryview doesn't implement the buffer protocol directly. However, it is a
wrapper around an object that implements the buffer protocol.

s* and y* value specifiers convert the type to a Py_buffer. This /should/
be more flexible than s#/y# and I believe it will accept memoryview
instances. Of course, we'd have to rewrite all our C code to use s* and y*.
Maybe we could just do it for bdiff as a start to prove it works.

It's worth noting that the Py_buffer interface is a bit more annoying to
code for. So we may not want to globally switch to it.


> +if pycompat.ispy3:
> +    def textdiff(a, b):
> +        return _textdiff(bytes(a), bytes(b))
> +else:
> +    textdiff = _textdiff
> +
>  class diffopts(object):
>      '''context is the number of context lines
>      text treats all files as text
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
Yuya Nishihara - March 3, 2018, 4:14 p.m.
On Sat, 3 Mar 2018 10:23:31 -0500, Gregory Szorc wrote:
> On Sat, Mar 3, 2018 at 8:27 AM, Yuya Nishihara <yuya@tcha.org> wrote:
> 
> > # HG changeset patch
> > # User Yuya Nishihara <yuya@tcha.org>
> > # Date 1520079865 18000
> > #      Sat Mar 03 07:24:25 2018 -0500
> > # Node ID 5e6c27a228b656f730cf0afd70e864ec96fc29c2
> > # Parent  d02adc818848486b23894e5fea4f632fb56b5aa4
> > py3: do not pass a memoryview to bdiff.bdiff()
> >
> > This doesn't look nice, but I don't know how to make a zero-copy slice of
> > bytes which is compatible with the buffer protocol.
> >
> > diff --git a/mercurial/mdiff.py b/mercurial/mdiff.py
> > --- a/mercurial/mdiff.py
> > +++ b/mercurial/mdiff.py
> > @@ -30,9 +30,17 @@ blocks = bdiff.blocks
> >  fixws = bdiff.fixws
> >  patches = mpatch.patches
> >  patchedsize = mpatch.patchedsize
> > -textdiff = bdiff.bdiff
> > +_textdiff = bdiff.bdiff
> >  splitnewlines = bdiff.splitnewlines
> >
> > +# On Python 3, util.buffer() creates a memoryview, which appears not
> > +# supporting the buffer protocol
> >
> 
> memoryview doesn't implement the buffer protocol directly. However, it is a
> wrapper around an object that implements the buffer protocol.
> 
> s* and y* value specifiers convert the type to a Py_buffer. This /should/
> be more flexible than s#/y# and I believe it will accept memoryview
> instances. Of course, we'd have to rewrite all our C code to use s* and y*.
> Maybe we could just do it for bdiff as a start to prove it works.
> 
> It's worth noting that the Py_buffer interface is a bit more annoying to
> code for. So we may not want to globally switch to it.

'y*' seems working. I'll send a patch.

Patch

diff --git a/mercurial/mdiff.py b/mercurial/mdiff.py
--- a/mercurial/mdiff.py
+++ b/mercurial/mdiff.py
@@ -30,9 +30,17 @@  blocks = bdiff.blocks
 fixws = bdiff.fixws
 patches = mpatch.patches
 patchedsize = mpatch.patchedsize
-textdiff = bdiff.bdiff
+_textdiff = bdiff.bdiff
 splitnewlines = bdiff.splitnewlines
 
+# On Python 3, util.buffer() creates a memoryview, which appears not
+# supporting the buffer protocol
+if pycompat.ispy3:
+    def textdiff(a, b):
+        return _textdiff(bytes(a), bytes(b))
+else:
+    textdiff = _textdiff
+
 class diffopts(object):
     '''context is the number of context lines
     text treats all files as text