Patchwork [3,of,3,RFC] mercurial: rewrite "iteritems" to "items"

login
register
mail settings
Submitter Gregory Szorc
Date May 16, 2016, 4:02 a.m.
Message ID <7e5fd80296fec5acd073.1463371373@ubuntu-vm-main>
Download mbox | patch
Permalink /patch/15147/
State RFC, archived
Headers show

Comments

Gregory Szorc - May 16, 2016, 4:02 a.m.
# HG changeset patch
# User Gregory Szorc <gregory.szorc@gmail.com>
# Date 1463371090 25200
#      Sun May 15 20:58:10 2016 -0700
# Node ID 7e5fd80296fec5acd073b166b8c26d3b2ee3d01b
# Parent  effc44c0ec7440fabb134a076e974ca478d89e11
mercurial: rewrite "iteritems" to "items"

This is extremely hacky. We should probably verify the token
before is an OP token with value ".". But it does result
in not encountering an iteritems run-time failure!

This change demonstrates we can pave over some Python 2/3
differences using on-the-fly source transformations without
"contaminating" the original source code to be Python 3
aware.

With this change, we fail on some attribute access in mail.py.
timeless - May 16, 2016, 3:27 p.m.
From my work I think we had some actual iteritems functions, I'd expect
this to do the wrong things for those cases.
On May 16, 2016 12:03 AM, "Gregory Szorc" <gregory.szorc@gmail.com> wrote:

> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1463371090 25200
> #      Sun May 15 20:58:10 2016 -0700
> # Node ID 7e5fd80296fec5acd073b166b8c26d3b2ee3d01b
> # Parent  effc44c0ec7440fabb134a076e974ca478d89e11
> mercurial: rewrite "iteritems" to "items"
>
> This is extremely hacky. We should probably verify the token
> before is an OP token with value ".". But it does result
> in not encountering an iteritems run-time failure!
>
> This change demonstrates we can pave over some Python 2/3
> differences using on-the-fly source transformations without
> "contaminating" the original source code to be Python 3
> aware.
>
> With this change, we fail on some attribute access in mail.py.
>
> diff --git a/mercurial/__init__.py b/mercurial/__init__.py
> --- a/mercurial/__init__.py
> +++ b/mercurial/__init__.py
> @@ -197,16 +197,23 @@ if sys.version_info[0] >= 3:
>                  return t
>
>              if s[0] not in ("'", '"'):
>                  return t
>
>              # String literal. Prefix to make a b'' string.
>              return tokenize.TokenInfo(t.type, 'b%s' % s, t.start, t.end,
> t.line)
>
> +        if t.type == token.NAME:
> +            if t.string == 'iteritems':
> +                return tokenize.TokenInfo(t.type, 'items', t.start, t.end,
> +                                          t.line)
> +
> +            return t
> +
>          return t
>
>      class hgloader(importlib.machinery.SourceFileLoader):
>          """Custom module loader that transforms source code.
>
>          When the source code is converted to code, we first transform
>          string literals to byte literals using the tokenize API.
>          """
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
Gregory Szorc - May 16, 2016, 3:51 p.m.
> On May 16, 2016, at 08:27, timeless <timeless@gmail.com> wrote:
> 
> From my work I think we had some actual iteritems functions, I'd expect this to do the wrong things for those cases.
> 
Right. We could also implement items() on our custom classes to conform with the Python 3 API.

>> On May 16, 2016 12:03 AM, "Gregory Szorc" <gregory.szorc@gmail.com> wrote:
>> # HG changeset patch
>> # User Gregory Szorc <gregory.szorc@gmail.com>
>> # Date 1463371090 25200
>> #      Sun May 15 20:58:10 2016 -0700
>> # Node ID 7e5fd80296fec5acd073b166b8c26d3b2ee3d01b
>> # Parent  effc44c0ec7440fabb134a076e974ca478d89e11
>> mercurial: rewrite "iteritems" to "items"
>> 
>> This is extremely hacky. We should probably verify the token
>> before is an OP token with value ".". But it does result
>> in not encountering an iteritems run-time failure!
>> 
>> This change demonstrates we can pave over some Python 2/3
>> differences using on-the-fly source transformations without
>> "contaminating" the original source code to be Python 3
>> aware.
>> 
>> With this change, we fail on some attribute access in mail.py.
>> 
>> diff --git a/mercurial/__init__.py b/mercurial/__init__.py
>> --- a/mercurial/__init__.py
>> +++ b/mercurial/__init__.py
>> @@ -197,16 +197,23 @@ if sys.version_info[0] >= 3:
>>                  return t
>> 
>>              if s[0] not in ("'", '"'):
>>                  return t
>> 
>>              # String literal. Prefix to make a b'' string.
>>              return tokenize.TokenInfo(t.type, 'b%s' % s, t.start, t.end, t.line)
>> 
>> +        if t.type == token.NAME:
>> +            if t.string == 'iteritems':
>> +                return tokenize.TokenInfo(t.type, 'items', t.start, t.end,
>> +                                          t.line)
>> +
>> +            return t
>> +
>>          return t
>> 
>>      class hgloader(importlib.machinery.SourceFileLoader):
>>          """Custom module loader that transforms source code.
>> 
>>          When the source code is converted to code, we first transform
>>          string literals to byte literals using the tokenize API.
>>          """
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Patch

diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -197,16 +197,23 @@  if sys.version_info[0] >= 3:
                 return t
 
             if s[0] not in ("'", '"'):
                 return t
 
             # String literal. Prefix to make a b'' string.
             return tokenize.TokenInfo(t.type, 'b%s' % s, t.start, t.end, t.line)
 
+        if t.type == token.NAME:
+            if t.string == 'iteritems':
+                return tokenize.TokenInfo(t.type, 'items', t.start, t.end,
+                                          t.line)
+
+            return t
+
         return t
 
     class hgloader(importlib.machinery.SourceFileLoader):
         """Custom module loader that transforms source code.
 
         When the source code is converted to code, we first transform
         string literals to byte literals using the tokenize API.
         """