Patchwork setup: use setuptools on Windows (issue5400)

login
register
mail settings
Submitter Matt Harbison
Date April 9, 2017, 12:30 a.m.
Message ID <op.yyejprvi9lwrgf@envy>
Download mbox | patch
Permalink /patch/20030/
State Not Applicable
Headers show

Comments

Matt Harbison - April 9, 2017, 12:30 a.m.
On Thu, 09 Mar 2017 23:06:34 -0500, Gregory Szorc  
<gregory.szorc@gmail.com> wrote:

> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1489118392 28800
> #      Thu Mar 09 19:59:52 2017 -0800
> # Node ID b51f9adb41e68d9f3d88582f044d2742ae29ce09
> # Parent  cd29673cebdbe2d998009322e4c3657389d6aed0
> setup: use setuptools on Windows (issue5400)
>
> We've had a long, complicated history with setuptools. We want to
> make it the universal default. But when we do, it breaks things.
>
> `python setup.py build` is broken on Windows today. Forcing
> the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things.
>
> Since the previous bustage with making setuptools the default
> was on !Windows, it seems safe to move ahead with the setuptools
> transition on Windows. So this patch does that.

I'm not sure why, or what we should do about it, but test-hghave.t fails  
with this on Windows.  I can get a similar failure on Linux with  
FORCE_SETUPTOOLS=1.  Is this an expected diff with setuptools (and  
deserving a '(glob) (?)'?

C:\Users\Matt\AppData\Local\Temp\hgtests.ie8k3m\install\lib\python\mercurial)
    # Ran 1 tests, 0 skipped, 0 warned, 0 failed.


> diff --git a/setup.py b/setup.py
> --- a/setup.py
> +++ b/setup.py
> @@ -63,7 +63,10 @@ import re
>  import shutil
>  import tempfile
>  from distutils import log
> -if 'FORCE_SETUPTOOLS' in os.environ:
> +# We have issues with setuptools on some platforms and builders. Until
> +# those are resolved, setuptools is opt-in except for platforms where
> +# we don't have issues.
> +if os.name == 'nt' or 'FORCE_SETUPTOOLS' in os.environ:
>      from setuptools import setup
>  else:
>      from distutils.core import setup
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Yuya Nishihara - April 9, 2017, 1:28 p.m.
On Sat, 08 Apr 2017 20:30:53 -0400, Matt Harbison wrote:
> On Thu, 09 Mar 2017 23:06:34 -0500, Gregory Szorc  
> <gregory.szorc@gmail.com> wrote:
> 
> > # HG changeset patch
> > # User Gregory Szorc <gregory.szorc@gmail.com>
> > # Date 1489118392 28800
> > #      Thu Mar 09 19:59:52 2017 -0800
> > # Node ID b51f9adb41e68d9f3d88582f044d2742ae29ce09
> > # Parent  cd29673cebdbe2d998009322e4c3657389d6aed0
> > setup: use setuptools on Windows (issue5400)
> >
> > We've had a long, complicated history with setuptools. We want to
> > make it the universal default. But when we do, it breaks things.
> >
> > `python setup.py build` is broken on Windows today. Forcing
> > the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things.
> >
> > Since the previous bustage with making setuptools the default
> > was on !Windows, it seems safe to move ahead with the setuptools
> > transition on Windows. So this patch does that.
> 
> I'm not sure why, or what we should do about it, but test-hghave.t fails  
> with this on Windows.  I can get a similar failure on Linux with  
> FORCE_SETUPTOOLS=1.  Is this an expected diff with setuptools (and  
> deserving a '(glob) (?)'?
> 
> --- c:/Users/Matt/Projects/hg/tests/test-hghave.t
> +++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
> @@ -19,7 +19,11 @@
>     >   foo
>     > EOF
>     $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
> +  warning: Testing with unexpected mercurial lib:  
> C:\Users\Matt\AppData\Local\Temp\hgtests.ie8k3minstall\lib\python\mercurial-4.1.1-py2.7-win-amd64.egg\mercurial
> +           (expected  
> C:\Users\Matt\AppData\Local\Temp\hgtests.ie8k3m\install\lib\python\mercurial)

Perhaps this assumption isn't true for setuptools.

https://www.mercurial-scm.org/repo/hg/file/4.1.2/tests/run-tests.py#l2469

I have no idea if setuptools can be configured to use simple installation
layout.
Matt Harbison - April 9, 2017, 8:40 p.m.
On Sun, 09 Apr 2017 09:28:04 -0400, Yuya Nishihara <yuya@tcha.org> wrote:

> On Sat, 08 Apr 2017 20:30:53 -0400, Matt Harbison wrote:
>> On Thu, 09 Mar 2017 23:06:34 -0500, Gregory Szorc
>> <gregory.szorc@gmail.com> wrote:
>>
>> > # HG changeset patch
>> > # User Gregory Szorc <gregory.szorc@gmail.com>
>> > # Date 1489118392 28800
>> > #      Thu Mar 09 19:59:52 2017 -0800
>> > # Node ID b51f9adb41e68d9f3d88582f044d2742ae29ce09
>> > # Parent  cd29673cebdbe2d998009322e4c3657389d6aed0
>> > setup: use setuptools on Windows (issue5400)
>> >
>> > We've had a long, complicated history with setuptools. We want to
>> > make it the universal default. But when we do, it breaks things.
>> >
>> > `python setup.py build` is broken on Windows today. Forcing
>> > the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things.
>> >
>> > Since the previous bustage with making setuptools the default
>> > was on !Windows, it seems safe to move ahead with the setuptools
>> > transition on Windows. So this patch does that.
>>
>> I'm not sure why, or what we should do about it, but test-hghave.t fails
>> with this on Windows.  I can get a similar failure on Linux with
>> FORCE_SETUPTOOLS=1.  Is this an expected diff with setuptools (and
>> deserving a '(glob) (?)'?
>>
>> --- c:/Users/Matt/Projects/hg/tests/test-hghave.t
>> +++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
>> @@ -19,7 +19,11 @@
>>     >   foo
>>     > EOF
>>     $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
>> +  warning: Testing with unexpected mercurial lib:
>> C:\Users\Matt\AppData\Local\Temp\hgtests.ie8k3minstall\lib\python\mercurial-4.1.1-py2.7-win-amd64.egg\mercurial
>> +           (expected
>> C:\Users\Matt\AppData\Local\Temp\hgtests.ie8k3m\install\lib\python\mercurial)
>
> Perhaps this assumption isn't true for setuptools.
>
> https://www.mercurial-scm.org/repo/hg/file/4.1.2/tests/run-tests.py#l2469
>
> I have no idea if setuptools can be configured to use simple installation
> layout.

That's a good tip, thanks.  The output above is from the original patch,  
but it is simpler for @ committed:

+  warning: Testing with unexpected mercurial lib:  
c:\Users\Matt\Projects\hg\mercurial
+           (expected  
C:\Users\Matt\AppData\Local\Temp\hgtests.q1ldmg\install\lib\python\mercurial)

Without setuptools:

expecthg:  
C:\Users\Matt\AppData\Local\Temp\hgtests.kshzjl\install\lib\python\mercurial
actualhg:  
C:\Users\Matt\AppData\Local\Temp\hgtests.kshzjl\install\lib\python\mercurial
bindir:   C:\Users\Matt\AppData\Local\Temp\hgtests.kshzjl\install\bin
pydir:     
C:\Users\Matt\AppData\Local\Temp\hgtests.kshzjl\install\lib\python
tmpdir:   C:\Users\Matt\AppData\Local\Temp\hgtests.kshzjl\install\bin

With setuptools:

expecthg:  
C:\Users\Matt\AppData\Local\Temp\hgtests.q1ldmg\install\lib\python\mercurial
actualhg: c:\Users\Matt\Projects\hg\mercurial
bindir:   C:\Users\Matt\AppData\Local\Temp\hgtests.q1ldmg\install\bin
pydir:     
C:\Users\Matt\AppData\Local\Temp\hgtests.q1ldmg\install\lib\python
tmpdir:   C:\Users\Matt\AppData\Local\Temp\hgtests.q1ldmg\install\bin

So it seems like the test is importing the 'hg' module that the main test  
runner is using?  It looks like test-run-tests.t papers over this by  
invoking run-tests.py with '--with-hg=`which hg`', which also works here.   
But someone who understands the underlying issue should take a look before  
doing that.  The function you referenced bails in this case, so it feels  
like subverting a valid check.
Pierre-Yves David - May 4, 2017, 12:12 a.m.
On 04/09/2017 02:30 AM, Matt Harbison wrote:
> On Thu, 09 Mar 2017 23:06:34 -0500, Gregory Szorc
> <gregory.szorc@gmail.com> wrote:
>
>> # HG changeset patch
>> # User Gregory Szorc <gregory.szorc@gmail.com>
>> # Date 1489118392 28800
>> #      Thu Mar 09 19:59:52 2017 -0800
>> # Node ID b51f9adb41e68d9f3d88582f044d2742ae29ce09
>> # Parent  cd29673cebdbe2d998009322e4c3657389d6aed0
>> setup: use setuptools on Windows (issue5400)
>>
>> We've had a long, complicated history with setuptools. We want to
>> make it the universal default. But when we do, it breaks things.
>>
>> `python setup.py build` is broken on Windows today. Forcing
>> the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things.
>>
>> Since the previous bustage with making setuptools the default
>> was on !Windows, it seems safe to move ahead with the setuptools
>> transition on Windows. So this patch does that.
>
> I'm not sure why, or what we should do about it, but test-hghave.t fails
> with this on Windows.  I can get a similar failure on Linux with
> FORCE_SETUPTOOLS=1.  Is this an expected diff with setuptools (and
> deserving a '(glob) (?)'?

It seems like your setuptools version/config installs '.egg'. That is 
awful since '.egg' is one of the worse things that happened to python 
packaging (their implementation is awful, has performance issue and 
break multiple basic assumption about python install/import).

I though the time were setuptools were intalling '.egg' were over. I 
wonder why your version installed '.eggs', is it some global 
configuration? Is it an older version? Am I wrong when I think '.egg' 
are no longer the default?

In all cases, I don't know if Mercurial setup tools can make sure it is 
never installed as an 'egg'. Maybe redefining the egg related commands 
in setup.py to abort ?

Cheers,
Matt Harbison - May 4, 2017, 1:09 a.m.
On Wed, 03 May 2017 20:12:39 -0400, Pierre-Yves David  
<pierre-yves.david@ens-lyon.org> wrote:

> On 04/09/2017 02:30 AM, Matt Harbison wrote:
>> On Thu, 09 Mar 2017 23:06:34 -0500, Gregory Szorc
>> <gregory.szorc@gmail.com> wrote:
>>
>>> # HG changeset patch
>>> # User Gregory Szorc <gregory.szorc@gmail.com>
>>> # Date 1489118392 28800
>>> #      Thu Mar 09 19:59:52 2017 -0800
>>> # Node ID b51f9adb41e68d9f3d88582f044d2742ae29ce09
>>> # Parent  cd29673cebdbe2d998009322e4c3657389d6aed0
>>> setup: use setuptools on Windows (issue5400)
>>>
>>> We've had a long, complicated history with setuptools. We want to
>>> make it the universal default. But when we do, it breaks things.
>>>
>>> `python setup.py build` is broken on Windows today. Forcing
>>> the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things.
>>>
>>> Since the previous bustage with making setuptools the default
>>> was on !Windows, it seems safe to move ahead with the setuptools
>>> transition on Windows. So this patch does that.
>>
>> I'm not sure why, or what we should do about it, but test-hghave.t fails
>> with this on Windows.  I can get a similar failure on Linux with
>> FORCE_SETUPTOOLS=1.  Is this an expected diff with setuptools (and
>> deserving a '(glob) (?)'?
>
> It seems like your setuptools version/config installs '.egg'. That is  
> awful since '.egg' is one of the worse things that happened to python  
> packaging (their implementation is awful, has performance issue and  
> break multiple basic assumption about python install/import).
>
> I though the time were setuptools were intalling '.egg' were over. I  
> wonder why your version installed '.eggs', is it some global  
> configuration? Is it an older version? Am I wrong when I think '.egg'  
> are no longer the default?
>
> In all cases, I don't know if Mercurial setup tools can make sure it is  
> never installed as an 'egg'. Maybe redefining the egg related commands  
> in setup.py to abort ?
>
> Cheers,
>

I've got python 2.7.13 installed.  `pip install --upgrade setuptools`  
bumped it from 28.8.0 -> 35.0.2, but no joy.  I deleted the contents of  
'mercurial.egg-info'.  `make clean && make local` doesn't repopulate the  
egg directory, but running the test does, and it fails again.

I can't believe that there's a global config in play on my system because  
I know nothing about setuptools, and Mercurial is the only python thing I  
work with.
Sean Farley - May 9, 2017, 12:08 a.m.
Matt Harbison <mharbison72@gmail.com> writes:

> On Wed, 03 May 2017 20:12:39 -0400, Pierre-Yves David  
> <pierre-yves.david@ens-lyon.org> wrote:
>
>> On 04/09/2017 02:30 AM, Matt Harbison wrote:
>>> On Thu, 09 Mar 2017 23:06:34 -0500, Gregory Szorc
>>> <gregory.szorc@gmail.com> wrote:
>>>
>>>> # HG changeset patch
>>>> # User Gregory Szorc <gregory.szorc@gmail.com>
>>>> # Date 1489118392 28800
>>>> #      Thu Mar 09 19:59:52 2017 -0800
>>>> # Node ID b51f9adb41e68d9f3d88582f044d2742ae29ce09
>>>> # Parent  cd29673cebdbe2d998009322e4c3657389d6aed0
>>>> setup: use setuptools on Windows (issue5400)
>>>>
>>>> We've had a long, complicated history with setuptools. We want to
>>>> make it the universal default. But when we do, it breaks things.
>>>>
>>>> `python setup.py build` is broken on Windows today. Forcing
>>>> the use of setuptools via FORCE_SETUPTOOLS=1 unbreaks things.
>>>>
>>>> Since the previous bustage with making setuptools the default
>>>> was on !Windows, it seems safe to move ahead with the setuptools
>>>> transition on Windows. So this patch does that.
>>>
>>> I'm not sure why, or what we should do about it, but test-hghave.t fails
>>> with this on Windows.  I can get a similar failure on Linux with
>>> FORCE_SETUPTOOLS=1.  Is this an expected diff with setuptools (and
>>> deserving a '(glob) (?)'?
>>
>> It seems like your setuptools version/config installs '.egg'. That is  
>> awful since '.egg' is one of the worse things that happened to python  
>> packaging (their implementation is awful, has performance issue and  
>> break multiple basic assumption about python install/import).
>>
>> I though the time were setuptools were intalling '.egg' were over. I  
>> wonder why your version installed '.eggs', is it some global  
>> configuration? Is it an older version? Am I wrong when I think '.egg'  
>> are no longer the default?
>>
>> In all cases, I don't know if Mercurial setup tools can make sure it is  
>> never installed as an 'egg'. Maybe redefining the egg related commands  
>> in setup.py to abort ?
>>
>> Cheers,
>>
>
> I've got python 2.7.13 installed.  `pip install --upgrade setuptools`  
> bumped it from 28.8.0 -> 35.0.2, but no joy.  I deleted the contents of  
> 'mercurial.egg-info'.  `make clean && make local` doesn't repopulate the  
> egg directory, but running the test does, and it fails again.
>
> I can't believe that there's a global config in play on my system because  
> I know nothing about setuptools, and Mercurial is the only python thing I  
> work with.

I usually get this directory when I run in a virtualenv and install the
package via 'pip install -e .'

Patch

--- c:/Users/Matt/Projects/hg/tests/test-hghave.t
+++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
@@ -19,7 +19,11 @@ 
    >   foo
    > EOF
    $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
+  warning: Testing with unexpected mercurial lib:  
C:\Users\Matt\AppData\Local\Temp\hgtests.ie8k3minstall\lib\python\mercurial-4.1.1-py2.7-win-amd64.egg\mercurial
+           (expected  
C:\Users\Matt\AppData\Local\Temp\hgtests.ie8k3m\install\lib\python\mercurial)
    .
+  warning: Tested with unexpected mercurial lib:  
C:\Users\Matt\AppData\Local\Temp\hgtests.ie8k3minstall\lib\python\mercurial-4.1.1-py2.7-win-amd64.egg\mercurial
+           (expected