Patchwork [2,of,2] setup: prevent setuptools from laying an egg

On Tue, 09 May 2017 14:23:18 -0400, Augie Fackler <raf@durin42.com> wrote:

> On Mon, May 8, 2017 at 12:39 AM, Matt Harbison <mharbison72@gmail.com>  
> wrote:
>> # HG changeset patch
>> # User Matt Harbison <matt_harbison@yahoo.com>
>> # Date 1494214143 14400
>> #      Sun May 07 23:29:03 2017 -0400
>> # Node ID 707683d56e5cbd3fe3453ddab9a57222b998f2af
>> # Parent  36d9a659b9d76837faaf73fde3f5c5455231c2f9
>> setup: prevent setuptools from laying an egg
>
> I just ran this patch by dstufft (the pip maintainer), and he strongly
> suspects (without having tested) that this will break `pip list` for
> Mercurial. Also, he said this:
>
> 14:21 < dstufft> durin42: I think if you do ``mkdir -p src && mv
> mercurial src/``
>                  and patch setup.py to add ``package_dir={"": "src"},``  
> it'll
>                  resolve that for you
> 14:22 < dstufft> durin42: the problem you're going to run into, is stuff  
> depends
>                  on that .egg-info, even though it says egg in the name  
> still,
>                  that's still a part of modern packaging, and the reason  
> why it
>                  was picking up local-source-tree hg is likely going to  
> be
>                  because python adds ``.`` to sys.path at the very front
>
> So we should probably find some other workaround here. :(

Bafflingly, an empty install class lets the test work, but does not  
prevent the generation of .egg-info:

                    b'compiler': compiler,
                    b'base': os.path.join(self._hgtmp, b"build"),

Alternately, adding --local to that test will cause the error to be  
skipped [1].  (It ends up using hg from the source tree.)  But I didn't do  
that, thinking that undermining the check was the wrong way to go.  I  
assume if we keep generating eggs, `make clean` should get rid of it.

Regarding your previous email about finding a way to not 'cd' back to the  
source tree, I don't think that's what is happening.  PYTHONPATH when hg  
is installed for the internal test is:

C:\Users\Matt\AppData\Local\Temp\hgtests.ijjckm\install\lib\python;$TESTTMP;c:\Users\Matt\Projects\hg\tests;c:\Users\Matt\Projects\hg;c:\Users\Matt\Projects\hg\tests;c:\Users\Matt\Projects\hg\tests

(Interesting, that repetition at the end.)

self._installdir for the internal test matches that first path.  I assume  
that when the egg is installed, there's nothing found there, and it simply  
falls back to the source tree.  When I use the empty install class and  
print 'actualhg' in that check, it _does_ use the temp installed package  
instead of falling back to the source tree.  So while the empty class is a  
bit of black magic, I think it keeps us where we are without setuptools.


[1]  
https://www.mercurial-scm.org/repo/hg/file/247bb7a2c492/tests/run-tests.py#l2553


>>
>> Previously, test-hghave.t was failing on Windows (and on Linux if
>> $FORCE_SETUPTOOLS was set) with the following:
>>
>>   --- c:/Users/Matt/Projects/hg/tests/test-hghave.t
>>   +++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
>>   @@ -19,7 +19,11 @@
>>      >   foo
>>      > EOF
>>      $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
>>   +  warning: Testing with unexpected mercurial lib:  
>> c:\Users\Matt\Projects\hg\mercurial
>>   +           (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
>>      .
>>   +  warning: Tested with unexpected mercurial lib:  
>> c:\Users\Matt\Projects\hg\mercurial
>>   +           (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
>>
>> Also, `make install` was creating an *.egg-info in the local Mercurial  
>> source
>> tree, which `make clean` wasn't deleting.
>>
>> Someone familiar with setuptools should take a close look at this.  I  
>> originally
>> registered a subclass of 'install_egg_info' with an empty run(), which  
>> took care
>> of the egg-info generated by `make install`.  Looking at the output of  
>> setup.py
>> as run by run-tests._installhg(), I saw that 'bdist_egg' was being  
>> run.  I
>> registered a similar empty class for that command, but the egg was still
>> produced.  But when I registered this 'install' subclass with nothing  
>> but a
>> run() that called the super class run, 'bdist_egg' was no longer being  
>> called,
>> and the test worked.  Subsequently dropping everything except the  
>> 'install'
>> subclass with a minimal delegating run() allowed the test to run  
>> (though the
>> setup log had output from running 'install_egg_info').
>>
>> To be on the safe side (and to avoid output about eggs when the empty  
>> subclasses
>> would have prevented any egg processing), I added a filter to  
>> get_sub_commands()
>> to filter out the egg related stuff.  Unlike the others, 'bdist_egg' is  
>> part of
>> setuptools, so registering a subclass and still working without  
>> setuptools
>> probably isn't very clean.
>>
>> diff --git a/setup.py b/setup.py
>> --- a/setup.py
>> +++ b/setup.py
>> @@ -77,6 +77,7 @@
>>  from distutils.command.build_ext import build_ext
>>  from distutils.command.build_py import build_py
>>  from distutils.command.build_scripts import build_scripts
>> +from distutils.command.install import install
>>  from distutils.command.install_lib import install_lib
>>  from distutils.command.install_scripts import install_scripts
>>  from distutils.spawn import spawn, find_executable
>> @@ -461,6 +462,12 @@
>>          dir = os.path.dirname(self.get_ext_fullpath('dummy'))
>>          return os.path.join(self.build_temp, dir, 'hg.exe')
>>
>> +class hginstall(install):
>> +    def get_sub_commands(self):
>> +        # Screen out egg related commands to prevent egg generation
>> +        excl = set(['install_egg_info', 'bdist_egg'])
>> +        return filter(lambda x: x not in excl,  
>> install.get_sub_commands(self))
>> +
>>  class hginstalllib(install_lib):
>>      '''
>>      This is a specialization of install_lib that replaces the  
>> copy_file used
>> @@ -572,6 +579,7 @@
>>              'build_py': hgbuildpy,
>>              'build_scripts': hgbuildscripts,
>>              'build_hgextindex': buildhgextindex,
>> +            'install': hginstall,
>>              'install_lib': hginstalllib,
>>              'install_scripts': hginstallscripts,
>>              'build_hgexe': buildhgexe,
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel