Patchwork [2,of,2,STABLE] setup: search for proper library name on MinGW/msys2

login
register
mail settings
Submitter Gregory Szorc
Date April 26, 2016, 11:25 p.m.
Message ID <a42a7f71b89a026572f8.1461713156@ubuntu-vm-main>
Download mbox | patch
Permalink /patch/14797/
State Superseded
Headers show

Comments

Gregory Szorc - April 26, 2016, 11:25 p.m.
# HG changeset patch
# User Gregory Szorc <gregory.szorc@gmail.com>
# Date 1461712815 25200
#      Tue Apr 26 16:20:15 2016 -0700
# Branch stable
# Node ID a42a7f71b89a026572f8ce798fa2c2bbaf6dea3a
# Parent  ba89075c616d56c3f1ce02c97103c6d0ff8aa006
setup: search for proper library name on MinGW/msys2

Attempting to build Mercurial from source using MinGW from
msys2 on Windows produces a hg.exe that attempts to load e.g.
python27.dll. MinGW prefixes its library name with "lib" and
adds a period between the major and minor versions. e.g.
"libpython2.7.dll."

Before this patch, hg.exe files in a MinGW environment would
either fail to find a Python DLL or would attempt to load a
non-MinGW DLL, which would summarily explode. Either way,
hg.exe wouldn't work.

This patch changes builds so hg.exe produced by MinGW will
search for "libpythonX.Y" instead of "pythonXY" in hopes that
MinGW will find a matching library.

It is certainly possible to build Mercurial in a MinGW environment
but want to load against a non-MinGW Python. This patch will
likely break that configuration. I build the Mercurial wheels in
cmd.exe (not MinGW) so this doesn't impact me. I'm not sure what
other packagers do on Windows. If we need the ability to build under
MinGW but targeting the traditional library name, we can likely
make that configurable.
Gregory Szorc - April 26, 2016, 11:36 p.m.
On Tue, Apr 26, 2016 at 4:25 PM, Gregory Szorc <gregory.szorc@gmail.com>
wrote:

> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1461712815 25200
> #      Tue Apr 26 16:20:15 2016 -0700
> # Branch stable
> # Node ID a42a7f71b89a026572f8ce798fa2c2bbaf6dea3a
> # Parent  ba89075c616d56c3f1ce02c97103c6d0ff8aa006
> setup: search for proper library name on MinGW/msys2
>
>
Give the lateness in the release cycle, I'd feel a lot better about this if
other Windows packagers weighed in.

We discovered this issue at Mozilla as part of trying to make Firefox build
with msys2 rather than our hacked together msys1-based environment (which
contains the official Windows Python distribution instead of the msys one).
With this patch, `pip install Mercurial` "just works" under msys2/MinGW and
doesn't produce a useless hg.exe.
Adrian Buehlmann - April 27, 2016, 12:50 p.m.
On 2016-04-27 01:36, Gregory Szorc wrote:
> On Tue, Apr 26, 2016 at 4:25 PM, Gregory Szorc <gregory.szorc@gmail.com
> <mailto:gregory.szorc@gmail.com>> wrote:
> 
>     # HG changeset patch
>     # User Gregory Szorc <gregory.szorc@gmail.com
>     <mailto:gregory.szorc@gmail.com>>
>     # Date 1461712815 25200
>     #      Tue Apr 26 16:20:15 2016 -0700
>     # Branch stable
>     # Node ID a42a7f71b89a026572f8ce798fa2c2bbaf6dea3a
>     # Parent  ba89075c616d56c3f1ce02c97103c6d0ff8aa006
>     setup: search for proper library name on MinGW/msys2
> 
> 
> Give the lateness in the release cycle, I'd feel a lot better about this
> if other Windows packagers weighed in.
> 
> We discovered this issue at Mozilla as part of trying to make Firefox
> build with msys2 rather than our hacked together msys1-based environment
> (which contains the official Windows Python distribution instead of the
> msys one). With this patch, `pip install Mercurial` "just works" under
> msys2/MinGW and doesn't produce a useless hg.exe.

I started exewrapper.c mainly for being able to run more tests of the
testsuite on Windows.

I think Steve doesn't use it for the hg.exe that he's building for the
*.msi packages for Windows. I think he uses py2exe for that, which puts
the python modules into a library.zip.

So I believe your changes to exewrapper.c are unlikely to affect Steve.
Yuya Nishihara - April 27, 2016, 1:35 p.m.
On Tue, 26 Apr 2016 16:25:56 -0700, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1461712815 25200
> #      Tue Apr 26 16:20:15 2016 -0700
> # Branch stable
> # Node ID a42a7f71b89a026572f8ce798fa2c2bbaf6dea3a
> # Parent  ba89075c616d56c3f1ce02c97103c6d0ff8aa006
> setup: search for proper library name on MinGW/msys2
> 
> Attempting to build Mercurial from source using MinGW from
> msys2 on Windows produces a hg.exe that attempts to load e.g.
> python27.dll. MinGW prefixes its library name with "lib" and
> adds a period between the major and minor versions. e.g.
> "libpython2.7.dll."
> 
> Before this patch, hg.exe files in a MinGW environment would
> either fail to find a Python DLL or would attempt to load a
> non-MinGW DLL, which would summarily explode. Either way,
> hg.exe wouldn't work.
> 
> This patch changes builds so hg.exe produced by MinGW will
> search for "libpythonX.Y" instead of "pythonXY" in hopes that
> MinGW will find a matching library.
> 
> It is certainly possible to build Mercurial in a MinGW environment
> but want to load against a non-MinGW Python. This patch will
> likely break that configuration. I build the Mercurial wheels in
> cmd.exe (not MinGW) so this doesn't impact me. I'm not sure what
> other packagers do on Windows. If we need the ability to build under
> MinGW but targeting the traditional library name, we can likely
> make that configurable.

I'm not a packager, but this will break my Windows test environment. I
use MSYS or MSYS2 with Windows Python because I just want a sane shell
and ssh into Windows.

If python is not a MSYS python, setup.py will generate a normal win32
executable, which should be linked to python27.dll.
Gregory Szorc - April 27, 2016, 4:19 p.m.
On Wed, Apr 27, 2016 at 6:35 AM, Yuya Nishihara <yuya@tcha.org> wrote:

> On Tue, 26 Apr 2016 16:25:56 -0700, Gregory Szorc wrote:
> > # HG changeset patch
> > # User Gregory Szorc <gregory.szorc@gmail.com>
> > # Date 1461712815 25200
> > #      Tue Apr 26 16:20:15 2016 -0700
> > # Branch stable
> > # Node ID a42a7f71b89a026572f8ce798fa2c2bbaf6dea3a
> > # Parent  ba89075c616d56c3f1ce02c97103c6d0ff8aa006
> > setup: search for proper library name on MinGW/msys2
> >
> > Attempting to build Mercurial from source using MinGW from
> > msys2 on Windows produces a hg.exe that attempts to load e.g.
> > python27.dll. MinGW prefixes its library name with "lib" and
> > adds a period between the major and minor versions. e.g.
> > "libpython2.7.dll."
> >
> > Before this patch, hg.exe files in a MinGW environment would
> > either fail to find a Python DLL or would attempt to load a
> > non-MinGW DLL, which would summarily explode. Either way,
> > hg.exe wouldn't work.
> >
> > This patch changes builds so hg.exe produced by MinGW will
> > search for "libpythonX.Y" instead of "pythonXY" in hopes that
> > MinGW will find a matching library.
> >
> > It is certainly possible to build Mercurial in a MinGW environment
> > but want to load against a non-MinGW Python. This patch will
> > likely break that configuration. I build the Mercurial wheels in
> > cmd.exe (not MinGW) so this doesn't impact me. I'm not sure what
> > other packagers do on Windows. If we need the ability to build under
> > MinGW but targeting the traditional library name, we can likely
> > make that configurable.
>
> I'm not a packager, but this will break my Windows test environment. I
> use MSYS or MSYS2 with Windows Python because I just want a sane shell
> and ssh into Windows.
>
> If python is not a MSYS python, setup.py will generate a normal win32
> executable, which should be linked to python27.dll.
>

OK. You and Adrian make good points. I'll work on a v2.

Patch

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -364,18 +364,24 @@  class buildhgexe(build_ext):
     description = 'compile hg.exe from mercurial/exewrapper.c'
 
     def build_extensions(self):
         if os.name != 'nt':
             return
         if isinstance(self.compiler, HackedMingw32CCompiler):
             self.compiler.compiler_so = self.compiler.compiler # no -mdll
             self.compiler.dll_libraries = [] # no -lmsrvc90
+        # The official CPython distribution on Windows uses e.g. python27 as
+        # the library name. MinGW uses e.g. libpython2.7.
+        if os.environ.get('MSYSTEM') in ('MINGW32', 'MINGW64'):
+            libpattern = 'libpython%d.%d.dll'
+        else:
+            libpattern = 'python%d%d.dll'
         hv = sys.hexversion
-        pythonlib = 'python%d%d.dll' % (hv >> 24, (hv >> 16) & 0xff)
+        pythonlib = libpattern % (hv >> 24, (hv >> 16) & 0xff)
         with open('mercurial/hgpythonlib.h', 'wb') as f:
             f.write('/* this file is autogenerated by setup.py */\n')
             f.write('#define HGPYTHONLIB "%s"\n' % pythonlib)
         objects = self.compiler.compile(['mercurial/exewrapper.c'],
                                          output_dir=self.build_temp)
         dir = os.path.dirname(self.get_ext_fullpath('dummy'))
         target = os.path.join(dir, 'hg')
         self.compiler.link_executable(objects, target,