Patchwork [7,of,8] branchmap: remove use of buffer() to support Python 2.6

login
register
mail settings
Submitter Gregory Szorc
Date May 13, 2017, 7:54 p.m.
Message ID <d407296a5efc97105840.1494705279@ubuntu-vm-main>
Download mbox | patch
Permalink /patch/20611/
State Accepted
Headers show

Comments

Gregory Szorc - May 13, 2017, 7:54 p.m.
# HG changeset patch
# User Gregory Szorc <gregory.szorc@gmail.com>
# Date 1494701888 25200
#      Sat May 13 11:58:08 2017 -0700
# Node ID d407296a5efc9710584087d23e4ff9293d7fb6ef
# Parent  016157858ae7e60681163ba47dca6ba822319dbe
branchmap: remove use of buffer() to support Python 2.6

The use of buffer() was added in 7359157b9e46 to support Python 2.6,
which we no longer support.
Katsunori FUJIWARA - June 8, 2017, 4:02 p.m.
At Sat, 13 May 2017 12:54:39 -0700,
Gregory Szorc wrote:
> 
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1494701888 25200
> #      Sat May 13 11:58:08 2017 -0700
> # Node ID d407296a5efc9710584087d23e4ff9293d7fb6ef
> # Parent  016157858ae7e60681163ba47dca6ba822319dbe
> branchmap: remove use of buffer() to support Python 2.6
> 
> The use of buffer() was added in 7359157b9e46 to support Python 2.6,
> which we no longer support.

For SupportedPythonVersions wiki page and release note of coming
Mercurial 4.3 :-)

This unpacking feature seems available with Python 2.7.4 (released at
2013-04-06) or later, as "fixing issue10212".

  https://hg.python.org/cpython/raw-file/v2.7.4/Misc/NEWS
  https://bugs.python.org/issue10212

I confirmed that this patch causes failure in this code path with
Python 2.7.3 at least.

> diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
> --- a/mercurial/branchmap.py
> +++ b/mercurial/branchmap.py
> @@ -406,8 +406,7 @@ class revbranchcache(object):
>  
>          # fast path: extract data from cache, use it if node is matching
>          reponode = changelog.node(rev)[:_rbcnodelen]
> -        cachenode, branchidx = unpack_from(
> -            _rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
> +        cachenode, branchidx = unpack_from(_rbcrecfmt, self._rbcrevs, rbcrevidx)
>          close = bool(branchidx & _rbccloseflag)
>          if close:
>              branchidx &= _rbcbranchidxmask
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Augie Fackler - June 8, 2017, 6:41 p.m.
> On Jun 8, 2017, at 12:02, FUJIWARA Katsunori <foozy@lares.dti.ne.jp> wrote:
> 
> At Sat, 13 May 2017 12:54:39 -0700,
> Gregory Szorc wrote:
>> 
>> # HG changeset patch
>> # User Gregory Szorc <gregory.szorc@gmail.com>
>> # Date 1494701888 25200
>> #      Sat May 13 11:58:08 2017 -0700
>> # Node ID d407296a5efc9710584087d23e4ff9293d7fb6ef
>> # Parent  016157858ae7e60681163ba47dca6ba822319dbe
>> branchmap: remove use of buffer() to support Python 2.6
>> 
>> The use of buffer() was added in 7359157b9e46 to support Python 2.6,
>> which we no longer support.
> 
> For SupportedPythonVersions wiki page and release note of coming
> Mercurial 4.3 :-)
> 
> This unpacking feature seems available with Python 2.7.4 (released at
> 2013-04-06) or later, as "fixing issue10212".
> 
>  https://hg.python.org/cpython/raw-file/v2.7.4/Misc/NEWS
>  https://bugs.python.org/issue10212
> 
> I confirmed that this patch causes failure in this code path with
> Python 2.7.3 at least.

I think I'm inclined to tell people that 2.7.4 is the minimum, given that we *really* want people on something that's 2.7.10 or thereabouts anyway so they get proper modern TLS.

> 
>> diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
>> --- a/mercurial/branchmap.py
>> +++ b/mercurial/branchmap.py
>> @@ -406,8 +406,7 @@ class revbranchcache(object):
>> 
>>         # fast path: extract data from cache, use it if node is matching
>>         reponode = changelog.node(rev)[:_rbcnodelen]
>> -        cachenode, branchidx = unpack_from(
>> -            _rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
>> +        cachenode, branchidx = unpack_from(_rbcrecfmt, self._rbcrevs, rbcrevidx)
>>         close = bool(branchidx & _rbccloseflag)
>>         if close:
>>             branchidx &= _rbcbranchidxmask
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> 
> -- 
> ----------------------------------------------------------------------
> [FUJIWARA Katsunori]                             foozy@lares.dti.ne.jp
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Gregory Szorc - June 8, 2017, 7:13 p.m.
On Thu, Jun 8, 2017 at 11:41 AM, Augie Fackler <raf@durin42.com> wrote:

>
> > On Jun 8, 2017, at 12:02, FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> wrote:
> >
> > At Sat, 13 May 2017 12:54:39 -0700,
> > Gregory Szorc wrote:
> >>
> >> # HG changeset patch
> >> # User Gregory Szorc <gregory.szorc@gmail.com>
> >> # Date 1494701888 25200
> >> #      Sat May 13 11:58:08 2017 -0700
> >> # Node ID d407296a5efc9710584087d23e4ff9293d7fb6ef
> >> # Parent  016157858ae7e60681163ba47dca6ba822319dbe
> >> branchmap: remove use of buffer() to support Python 2.6
> >>
> >> The use of buffer() was added in 7359157b9e46 to support Python 2.6,
> >> which we no longer support.
> >
> > For SupportedPythonVersions wiki page and release note of coming
> > Mercurial 4.3 :-)
> >
> > This unpacking feature seems available with Python 2.7.4 (released at
> > 2013-04-06) or later, as "fixing issue10212".
> >
> >  https://hg.python.org/cpython/raw-file/v2.7.4/Misc/NEWS
> >  https://bugs.python.org/issue10212
> >
> > I confirmed that this patch causes failure in this code path with
> > Python 2.7.3 at least.
>
> I think I'm inclined to tell people that 2.7.4 is the minimum, given that
> we *really* want people on something that's 2.7.10 or thereabouts anyway so
> they get proper modern TLS.
>

Firefox's build system has required Python 2.7.3+ since September 2013 (
https://bugzilla.mozilla.org/show_bug.cgi?id=870420) (because we were tired
of working around bugs in older Python versions). AFAIK nobody has
complained about it in ages. So my experience says Mercurial can also get
away with requiring a minimum version of 2.7. In reality, this should only
impact Linux and Unixen as OS X and Windows are taken care of via our
packaging or Homebrew/MacPorts.


>
> >
> >> diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
> >> --- a/mercurial/branchmap.py
> >> +++ b/mercurial/branchmap.py
> >> @@ -406,8 +406,7 @@ class revbranchcache(object):
> >>
> >>         # fast path: extract data from cache, use it if node is matching
> >>         reponode = changelog.node(rev)[:_rbcnodelen]
> >> -        cachenode, branchidx = unpack_from(
> >> -            _rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
> >> +        cachenode, branchidx = unpack_from(_rbcrecfmt, self._rbcrevs,
> rbcrevidx)
> >>         close = bool(branchidx & _rbccloseflag)
> >>         if close:
> >>             branchidx &= _rbcbranchidxmask
> >> _______________________________________________
> >> Mercurial-devel mailing list
> >> Mercurial-devel@mercurial-scm.org
> >> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> >
> > --
> > ----------------------------------------------------------------------
> > [FUJIWARA Katsunori]                             foozy@lares.dti.ne.jp
> > _______________________________________________
> > Mercurial-devel mailing list
> > Mercurial-devel@mercurial-scm.org
> > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
>
Katsunori FUJIWARA - July 12, 2017, 7 p.m.
At Thu, 8 Jun 2017 12:13:27 -0700,
Gregory Szorc wrote:
> 
> [1  <text/plain; UTF-8 (7bit)>]
> On Thu, Jun 8, 2017 at 11:41 AM, Augie Fackler <raf@durin42.com> wrote:
> 
> >
> > > On Jun 8, 2017, at 12:02, FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> > wrote:
> > >
> > > At Sat, 13 May 2017 12:54:39 -0700,
> > > Gregory Szorc wrote:
> > >>
> > >> # HG changeset patch
> > >> # User Gregory Szorc <gregory.szorc@gmail.com>
> > >> # Date 1494701888 25200
> > >> #      Sat May 13 11:58:08 2017 -0700
> > >> # Node ID d407296a5efc9710584087d23e4ff9293d7fb6ef
> > >> # Parent  016157858ae7e60681163ba47dca6ba822319dbe
> > >> branchmap: remove use of buffer() to support Python 2.6
> > >>
> > >> The use of buffer() was added in 7359157b9e46 to support Python 2.6,
> > >> which we no longer support.
> > >
> > > For SupportedPythonVersions wiki page and release note of coming
> > > Mercurial 4.3 :-)
> > >
> > > This unpacking feature seems available with Python 2.7.4 (released at
> > > 2013-04-06) or later, as "fixing issue10212".
> > >
> > >  https://hg.python.org/cpython/raw-file/v2.7.4/Misc/NEWS
> > >  https://bugs.python.org/issue10212
> > >
> > > I confirmed that this patch causes failure in this code path with
> > > Python 2.7.3 at least.
> >
> > I think I'm inclined to tell people that 2.7.4 is the minimum, given that
> > we *really* want people on something that's 2.7.10 or thereabouts anyway so
> > they get proper modern TLS.
> >

I found one more breakage point.

Python 2.7.4 - 2.7.9 fails at creating zip archive on hgweb server
side, because Python zipfile module tries to invoke "seek" method on
wsgirequest object (defined in mercurial/hgweb/request.py), even
though it doesn't provide it.

This can reproduce by test-archive.t.

This unintentional "seek" invocation is recognized as issue14099 of
Python, and fixed at 2.7.10.

    https://bugs.python.org/issue14099

So, let's require 2.7.10 or later :-)

> Firefox's build system has required Python 2.7.3+ since September 2013 (
> https://bugzilla.mozilla.org/show_bug.cgi?id=870420) (because we were tired
> of working around bugs in older Python versions). AFAIK nobody has
> complained about it in ages. So my experience says Mercurial can also get
> away with requiring a minimum version of 2.7. In reality, this should only
> impact Linux and Unixen as OS X and Windows are taken care of via our
> packaging or Homebrew/MacPorts.

We need to update files below ?

  contrib/debian/control
  contrib/mercurial.spec

> >
> > >
> > >> diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
> > >> --- a/mercurial/branchmap.py
> > >> +++ b/mercurial/branchmap.py
> > >> @@ -406,8 +406,7 @@ class revbranchcache(object):
> > >>
> > >>         # fast path: extract data from cache, use it if node is matching
> > >>         reponode = changelog.node(rev)[:_rbcnodelen]
> > >> -        cachenode, branchidx = unpack_from(
> > >> -            _rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
> > >> +        cachenode, branchidx = unpack_from(_rbcrecfmt, self._rbcrevs,
> > rbcrevidx)
> > >>         close = bool(branchidx & _rbccloseflag)
> > >>         if close:
> > >>             branchidx &= _rbcbranchidxmask
> > >> _______________________________________________
> > >> Mercurial-devel mailing list
> > >> Mercurial-devel@mercurial-scm.org
> > >> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> > >
> > > --
> > > ----------------------------------------------------------------------
> > > [FUJIWARA Katsunori]                             foozy@lares.dti.ne.jp
> > > _______________________________________________
> > > Mercurial-devel mailing list
> > > Mercurial-devel@mercurial-scm.org
> > > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> >
> >
> [2  <text/html; UTF-8 (quoted-printable)>]
>
Gregory Szorc - July 12, 2017, 7:23 p.m.
On Wed, Jul 12, 2017 at 12:00 PM, FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
wrote:

> At Thu, 8 Jun 2017 12:13:27 -0700,
> Gregory Szorc wrote:
> >
> > [1  <text/plain; UTF-8 (7bit)>]
> > On Thu, Jun 8, 2017 at 11:41 AM, Augie Fackler <raf@durin42.com> wrote:
> >
> > >
> > > > On Jun 8, 2017, at 12:02, FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> > > wrote:
> > > >
> > > > At Sat, 13 May 2017 12:54:39 -0700,
> > > > Gregory Szorc wrote:
> > > >>
> > > >> # HG changeset patch
> > > >> # User Gregory Szorc <gregory.szorc@gmail.com>
> > > >> # Date 1494701888 25200
> > > >> #      Sat May 13 11:58:08 2017 -0700
> > > >> # Node ID d407296a5efc9710584087d23e4ff9293d7fb6ef
> > > >> # Parent  016157858ae7e60681163ba47dca6ba822319dbe
> > > >> branchmap: remove use of buffer() to support Python 2.6
> > > >>
> > > >> The use of buffer() was added in 7359157b9e46 to support Python 2.6,
> > > >> which we no longer support.
> > > >
> > > > For SupportedPythonVersions wiki page and release note of coming
> > > > Mercurial 4.3 :-)
> > > >
> > > > This unpacking feature seems available with Python 2.7.4 (released at
> > > > 2013-04-06) or later, as "fixing issue10212".
> > > >
> > > >  https://hg.python.org/cpython/raw-file/v2.7.4/Misc/NEWS
> > > >  https://bugs.python.org/issue10212
> > > >
> > > > I confirmed that this patch causes failure in this code path with
> > > > Python 2.7.3 at least.
> > >
> > > I think I'm inclined to tell people that 2.7.4 is the minimum, given
> that
> > > we *really* want people on something that's 2.7.10 or thereabouts
> anyway so
> > > they get proper modern TLS.
> > >
>
> I found one more breakage point.
>
> Python 2.7.4 - 2.7.9 fails at creating zip archive on hgweb server
> side, because Python zipfile module tries to invoke "seek" method on
> wsgirequest object (defined in mercurial/hgweb/request.py), even
> though it doesn't provide it.
>
> This can reproduce by test-archive.t.
>
> This unintentional "seek" invocation is recognized as issue14099 of
> Python, and fixed at 2.7.10.
>
>     https://bugs.python.org/issue14099


Ugh.


>
>
> So, let's require 2.7.10 or later :-)
>

I wish. Unfortunately that won't fly since RHEL/CentOS 7 ship 2.7.5 (albeit
a 2.7.5 with tons of patches, including backport of ssl niceness).


>
> > Firefox's build system has required Python 2.7.3+ since September 2013 (
> > https://bugzilla.mozilla.org/show_bug.cgi?id=870420) (because we were
> tired
> > of working around bugs in older Python versions). AFAIK nobody has
> > complained about it in ages. So my experience says Mercurial can also get
> > away with requiring a minimum version of 2.7. In reality, this should
> only
> > impact Linux and Unixen as OS X and Windows are taken care of via our
> > packaging or Homebrew/MacPorts.
>
> We need to update files below ?
>
>   contrib/debian/control
>   contrib/mercurial.spec
>
> > >
> > > >
> > > >> diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
> > > >> --- a/mercurial/branchmap.py
> > > >> +++ b/mercurial/branchmap.py
> > > >> @@ -406,8 +406,7 @@ class revbranchcache(object):
> > > >>
> > > >>         # fast path: extract data from cache, use it if node is
> matching
> > > >>         reponode = changelog.node(rev)[:_rbcnodelen]
> > > >> -        cachenode, branchidx = unpack_from(
> > > >> -            _rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
> > > >> +        cachenode, branchidx = unpack_from(_rbcrecfmt,
> self._rbcrevs,
> > > rbcrevidx)
> > > >>         close = bool(branchidx & _rbccloseflag)
> > > >>         if close:
> > > >>             branchidx &= _rbcbranchidxmask
> > > >> _______________________________________________
> > > >> Mercurial-devel mailing list
> > > >> Mercurial-devel@mercurial-scm.org
> > > >> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> > > >
> > > > --
> > > > ------------------------------------------------------------
> ----------
> > > > [FUJIWARA Katsunori]
> foozy@lares.dti.ne.jp
> > > > _______________________________________________
> > > > Mercurial-devel mailing list
> > > > Mercurial-devel@mercurial-scm.org
> > > > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> > >
> > >
> > [2  <text/html; UTF-8 (quoted-printable)>]
> >
>
> --
> ----------------------------------------------------------------------
> [FUJIWARA Katsunori]                             foozy@lares.dti.ne.jp
>

Patch

diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -406,8 +406,7 @@  class revbranchcache(object):
 
         # fast path: extract data from cache, use it if node is matching
         reponode = changelog.node(rev)[:_rbcnodelen]
-        cachenode, branchidx = unpack_from(
-            _rbcrecfmt, util.buffer(self._rbcrevs), rbcrevidx)
+        cachenode, branchidx = unpack_from(_rbcrecfmt, self._rbcrevs, rbcrevidx)
         close = bool(branchidx & _rbccloseflag)
         if close:
             branchidx &= _rbcbranchidxmask