Patchwork [11,of,22] buildrpm: introduce --withpython for building rpms that includes Python 2.7

login
register
mail settings
Submitter Mads Kiilerich
Date May 20, 2014, 2:10 a.m.
Message ID <b71a9c09bdc9b2a04679.1400551806@mk-desktop>
Download mbox | patch
Permalink /patch/4833/
State Superseded
Commit 96d95b75a63beaf62743c7fed32f443edff159c5
Headers show

Comments

Mads Kiilerich - May 20, 2014, 2:10 a.m.
# HG changeset patch
# User Mads Kiilerich <madski@unity3d.com>
# Date 1400111411 -7200
#      Thu May 15 01:50:11 2014 +0200
# Node ID b71a9c09bdc9b2a04679a34e0ef0765ffbbbb933
# Parent  528a57d3359489b137301ffca0e125cc1c0a84c3
buildrpm: introduce --withpython for building rpms that includes Python 2.7

Also available as "make rpm-py".
Simon King - May 20, 2014, 9 a.m.
On Tue, May 20, 2014 at 3:10 AM, Mads Kiilerich <mads@kiilerich.com> wrote:
> # HG changeset patch
> # User Mads Kiilerich <madski@unity3d.com>
> # Date 1400111411 -7200
> #      Thu May 15 01:50:11 2014 +0200
> # Node ID b71a9c09bdc9b2a04679a34e0ef0765ffbbbb933
> # Parent  528a57d3359489b137301ffca0e125cc1c0a84c3
> buildrpm: introduce --withpython for building rpms that includes Python 2.7
>
> Also available as "make rpm-py".
>
> diff --git a/contrib/buildrpm b/contrib/buildrpm
> --- a/contrib/buildrpm
> +++ b/contrib/buildrpm
> @@ -27,6 +27,10 @@ while [ "$1" ]; do
>          SUFFIX="$1"
>          shift
>          ;;
> +    --withpython )
> +        shift
> +        PYTHONVER=2.7.6
> +        ;;

I'm not at all familiar with building RPMs, but should that be "$1",
or does it intentionally only work for a single Python version?

Simon
Simon King - May 20, 2014, 9:04 a.m.
On Tue, May 20, 2014 at 10:00 AM, Simon King <simon@simonking.org.uk> wrote:
> On Tue, May 20, 2014 at 3:10 AM, Mads Kiilerich <mads@kiilerich.com> wrote:
>> # HG changeset patch
>> # User Mads Kiilerich <madski@unity3d.com>
>> # Date 1400111411 -7200
>> #      Thu May 15 01:50:11 2014 +0200
>> # Node ID b71a9c09bdc9b2a04679a34e0ef0765ffbbbb933
>> # Parent  528a57d3359489b137301ffca0e125cc1c0a84c3
>> buildrpm: introduce --withpython for building rpms that includes Python 2.7
>>
>> Also available as "make rpm-py".
>>
>> diff --git a/contrib/buildrpm b/contrib/buildrpm
>> --- a/contrib/buildrpm
>> +++ b/contrib/buildrpm
>> @@ -27,6 +27,10 @@ while [ "$1" ]; do
>>          SUFFIX="$1"
>>          shift
>>          ;;
>> +    --withpython )
>> +        shift
>> +        PYTHONVER=2.7.6
>> +        ;;
>
> I'm not at all familiar with building RPMs, but should that be "$1",
> or does it intentionally only work for a single Python version?
>

Sorry, I read more carefully and realised you are downloading and
building the latest python release, not reusing the version installed
on the system. Ignore me :-)

Simon
Olle Lundberg - May 20, 2014, 5:04 p.m.
On Tue, May 20, 2014 at 4:10 AM, Mads Kiilerich <mads@kiilerich.com> wrote:

> # HG changeset patch
> # User Mads Kiilerich <madski@unity3d.com>
> # Date 1400111411 -7200
> #      Thu May 15 01:50:11 2014 +0200
> # Node ID b71a9c09bdc9b2a04679a34e0ef0765ffbbbb933
> # Parent  528a57d3359489b137301ffca0e125cc1c0a84c3
> buildrpm: introduce --withpython for building rpms that includes Python 2.7
>
> Also available as "make rpm-py".
>
> diff --git a/contrib/buildrpm b/contrib/buildrpm
> --- a/contrib/buildrpm
> +++ b/contrib/buildrpm
> @@ -27,6 +27,10 @@ while [ "$1" ]; do
>          SUFFIX="$1"
>          shift
>          ;;
> +    --withpython )
> +        shift
> +        PYTHONVER=2.7.6
> +        ;;
>      --rpmbuilddir )
>          shift
>          RPMBUILDDIR="$1"
> @@ -45,7 +49,7 @@ done
>
>  cd "`dirname $0`/.."
>
> -specfile=contrib/mercurial.spec
> +specfile=$PWD/contrib/mercurial.spec
>  if [ ! -f $specfile ]; then
>      echo "Cannot find $specfile!" 1>&2
>      exit 1
> @@ -85,12 +89,31 @@ hgversion=`$HG log -r "$REV" -T '{ifeq(b
>
>  version=${hgversion%-*}
>  release=${hgversion#*-}
> +if [ "$PYTHONVER" ]; then
> +    release=$release+$PYTHONVER
> +fi
>
>  $HG archive -r "$REV" -t tgz
> $RPMBUILDDIR/SOURCES/mercurial-$version-$release.tar.gz
> +if [ "$PYTHONVER" ]; then
> +(
> +    cd build
> +    PYTHON_SRCFILE=Python-$PYTHONVER.tgz
> +    [ -f $PYTHON_SRCFILE ] || curl -Lo $PYTHON_SRCFILE
> http://www.python.org/ftp/python/$PYTHONVER/$PYTHON_SRCFILE
> +    ln -f $PYTHON_SRCFILE $RPMBUILDDIR/SOURCES/$PYTHON_SRCFILE
> +
> +    DOCUTILSVER=`sed -ne "s/^%global docutilsname docutils-//p" $specfile`
> +    DOCUTILS_SRCFILE=docutils-$DOCUTILSVER.tar.gz
> +    [ -f $DOCUTILS_SRCFILE ] || curl -Lo $DOCUTILS_SRCFILE
> http://downloads.sourceforge.net/project/docutils/docutils/$DOCUTILSVER/$DOCUTILS_SRCFILE
> +    ln -f $DOCUTILS_SRCFILE $RPMBUILDDIR/SOURCES/$DOCUTILS_SRCFILE
> +)
> +fi
> +
>  rpmspec=$RPMBUILDDIR/SPECS/mercurial.spec
>
>  (
>  sed \
> +    -e "s/^%define withpython.*$/%define withpython $PYTHONVER/" \
> +    -e "s/^%define withpython *$/%define withpython %{nil}/" \
>      -e "s,^Name:.*,Name: mercurial$SUFFIX," \
>      -e "s,^Version:.*,Version: $version," \
>      -e "s,^Release:.*,Release: $release," \
> diff --git a/contrib/mercurial.spec b/contrib/mercurial.spec
> --- a/contrib/mercurial.spec
> +++ b/contrib/mercurial.spec
> @@ -1,6 +1,23 @@
>  %global emacs_lispdir %{_datadir}/emacs/site-lisp
> +
> +%define withpython %{nil}
> +
> +%if "%{?withpython}"
> +
> +%global pythonver %{withpython}
> +%global pythonname Python-%{withpython}
> +%global docutilsname docutils-0.11
> +%global pythonhg python-hg
> +%global hgpyprefix /usr/%{pythonhg}
> +# byte compilation will fail on some some Python /test/ files
> +%global _python_bytecompile_errors_terminate_build 0
> +
> +%else
> +
>  %global pythonver %(python -c 'import sys;print ".".join(map(str,
> sys.version_info[:2]))')
>
> +%endif
> +
>  Summary: A fast, lightweight Source Control Management system
>  Name: mercurial
>  Version: snapshot
> @@ -9,11 +26,20 @@ License: GPLv2+
>  Group: Development/Tools
>  URL: http://mercurial.selenic.com/
>  Source0: mercurial-%{version}-%{release}.tar.gz
> +%if "%{?withpython}"
> +Source1: %{pythonname}.tgz
> +Source2: %{docutilsname}.tar.gz
> +%endif
>  BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
>
> -BuildRequires: python >= 2.4, python-devel, make, gcc, python-docutils >=
> 0.5, gettext
> -Provides: hg = %{version}-%{release}
> +BuildRequires: make, gcc, gettext
> +%if "%{?withpython}"
> +BuildRequires: readline-devel, openssl-devel, ncurses-devel, zlib-devel,
> bzip2-devel
> +%else
> +BuildRequires: python >= 2.4, python-devel, python-docutils >= 0.5
>  Requires: python >= 2.4
> +%endif
> +Provides: hg = %{version}-%{release}, mercurial = %{version}-%{release}
>  # The hgk extension uses the wish tcl interpreter, but we don't enforce it
>  #Requires: tk
>
> @@ -22,15 +48,67 @@ Mercurial is a fast, lightweight source
>  for efficient handling of very large distributed projects.
>
>  %prep
> +
> +%if "%{?withpython}"
> +%setup -q -n mercurial-%{version}-%{release} -a1 -a2
> +# despite the comments in cgi.py, we do this to prevent rpmdeps from
> picking /usr/local/bin/python up
> +sed -i '1c#! /usr/bin/env python' %{pythonname}/Lib/cgi.py
> +%else
>  %setup -q -n mercurial-%{version}-%{release}
> +%endif
>
>  %build
> +%if "%{?withpython}"
> +
> +PYPATH=$PWD/%{pythonname}
> +cd $PYPATH
> +./configure --prefix=%{hgpyprefix}
> +make all %{?_smp_mflags}
> +cd -
> +
> +cd %{docutilsname}
> +LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py build
> +cd -
> +
> +# verify Python environment
> +LD_LIBRARY_PATH=$PYPATH PYTHONPATH=$PWD/%{docutilsname} $PYPATH/python -c
> 'import sys, zlib, bz2, ssl, curses, readline'
> +
> +export PATH=$PYPATH:$PATH
> +export LD_LIBRARY_PATH=$PYPATH
> +export CFLAGS="-L $PYPATH"
> +export PYTHONPATH=$PWD/%{docutilsname}
> +
> +%endif
> +
>  make all
>
>  %install
>  rm -rf $RPM_BUILD_ROOT
> +
> +%if "%{?withpython}"
> +
> +PYPATH=$PWD/%{pythonname}
> +cd $PYPATH
> +make install DESTDIR=$RPM_BUILD_ROOT
> +# these .a are not necessary and they are readonly and strip fails - kill
> them!
> +rm -f %{buildroot}%{hgpyprefix}/lib/{,python2.*/config}/libpython2.*.a
> +cd -
> +
> +cd %{docutilsname}
> +LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py install
> --root="$RPM_BUILD_ROOT"
> +cd -
> +
> +PATH=$PYPATH:$PATH LD_LIBRARY_PATH=$PYPATH make install
> DESTDIR=$RPM_BUILD_ROOT PREFIX=%{hgpyprefix} MANDIR=%{_mandir}
> +mkdir -p $RPM_BUILD_ROOT%{_bindir}
> +( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/hg . )
> +( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/python2.?
> %{pythonhg} )
> +
> +%else
> +
>  make install DESTDIR=$RPM_BUILD_ROOT PREFIX=%{_prefix} MANDIR=%{_mandir}
>
> +%endif
> +
>  install -m 755 contrib/hgk $RPM_BUILD_ROOT%{_bindir}/
>  install -m 755 contrib/hg-ssh $RPM_BUILD_ROOT%{_bindir}/
>
> @@ -71,10 +149,15 @@ rm -rf $RPM_BUILD_ROOT
>  %dir %{_sysconfdir}/mercurial
>  %dir %{_sysconfdir}/mercurial/hgrc.d
>  %config(noreplace) %{_sysconfdir}/mercurial/hgrc.d/mergetools.rc
> +%if "%{?withpython}"
> +%{_bindir}/%{pythonhg}
> +%{hgpyprefix}
> +%else
>  %if "%{?pythonver}" != "2.4"
>
>  %{_libdir}/python%{pythonver}/site-packages/mercurial-*-py%{pythonver}.egg-info
>  %endif
>  %{_libdir}/python%{pythonver}/site-packages/mercurial
>  %{_libdir}/python%{pythonver}/site-packages/hgext
> +%endif
>
>  %changelog
>

If you use the --with/--without flag to rpmbuild and use the corresponding
%bcond_with/%bcond_without macros, this would be cleaner, you would also
not need the initial sed.

More information about conditionals here:
http://www.rpm.org/wiki/PackagerDocs/ConditionalBuilds


> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
>
Mads Kiilerich - May 20, 2014, 6:38 p.m.
On 05/20/2014 07:04 PM, Olle wrote:
>
> @@ -71,10 +149,15 @@ rm -rf $RPM_BUILD_ROOT
>
>      %dir %{_sysconfdir}/mercurial
>      %dir %{_sysconfdir}/mercurial/hgrc.d
>      %config(noreplace) %{_sysconfdir}/mercurial/hgrc.d/mergetools.rc
>     +%if "%{?withpython}"
>     +%{_bindir}/%{pythonhg}
>     +%{hgpyprefix}
>     +%else
>      %if "%{?pythonver}" != "2.4"
>      %{_libdir}/python%{pythonver}/site-packages/mercurial-*-py%{pythonver}.egg-info
>      %endif
>      %{_libdir}/python%{pythonver}/site-packages/mercurial
>      %{_libdir}/python%{pythonver}/site-packages/hgext
>     +%endif
>
>      %changelog
>
>
> If you use the --with/--without flag to rpmbuild and use the 
> corresponding %bcond_with/%bcond_without macros, this would be 
> cleaner, you would also not need the initial sed.

Yes, it could be done in many different ways. This is an extension of 
the existing way of doing it.

One advantage of doing it this way is that the srpm just can be rebuild 
and will do what it is intended to do. We also don't want to include 
Python in srpms where it isn't needed and can't build with Python if it 
isn't included.

We also "have" to modify the spec in order to set the version and add a 
changelog.

/Mads

Patch

diff --git a/contrib/buildrpm b/contrib/buildrpm
--- a/contrib/buildrpm
+++ b/contrib/buildrpm
@@ -27,6 +27,10 @@  while [ "$1" ]; do
         SUFFIX="$1"
         shift
         ;;
+    --withpython )
+        shift
+        PYTHONVER=2.7.6
+        ;;
     --rpmbuilddir )
         shift
         RPMBUILDDIR="$1"
@@ -45,7 +49,7 @@  done
 
 cd "`dirname $0`/.."
 
-specfile=contrib/mercurial.spec
+specfile=$PWD/contrib/mercurial.spec
 if [ ! -f $specfile ]; then
     echo "Cannot find $specfile!" 1>&2
     exit 1
@@ -85,12 +89,31 @@  hgversion=`$HG log -r "$REV" -T '{ifeq(b
 
 version=${hgversion%-*}
 release=${hgversion#*-}
+if [ "$PYTHONVER" ]; then
+    release=$release+$PYTHONVER
+fi
 
 $HG archive -r "$REV" -t tgz $RPMBUILDDIR/SOURCES/mercurial-$version-$release.tar.gz
+if [ "$PYTHONVER" ]; then
+(
+    cd build
+    PYTHON_SRCFILE=Python-$PYTHONVER.tgz
+    [ -f $PYTHON_SRCFILE ] || curl -Lo $PYTHON_SRCFILE http://www.python.org/ftp/python/$PYTHONVER/$PYTHON_SRCFILE
+    ln -f $PYTHON_SRCFILE $RPMBUILDDIR/SOURCES/$PYTHON_SRCFILE
+
+    DOCUTILSVER=`sed -ne "s/^%global docutilsname docutils-//p" $specfile`
+    DOCUTILS_SRCFILE=docutils-$DOCUTILSVER.tar.gz
+    [ -f $DOCUTILS_SRCFILE ] || curl -Lo $DOCUTILS_SRCFILE http://downloads.sourceforge.net/project/docutils/docutils/$DOCUTILSVER/$DOCUTILS_SRCFILE
+    ln -f $DOCUTILS_SRCFILE $RPMBUILDDIR/SOURCES/$DOCUTILS_SRCFILE
+)
+fi
+
 rpmspec=$RPMBUILDDIR/SPECS/mercurial.spec
 
 (
 sed \
+    -e "s/^%define withpython.*$/%define withpython $PYTHONVER/" \
+    -e "s/^%define withpython *$/%define withpython %{nil}/" \
     -e "s,^Name:.*,Name: mercurial$SUFFIX," \
     -e "s,^Version:.*,Version: $version," \
     -e "s,^Release:.*,Release: $release," \
diff --git a/contrib/mercurial.spec b/contrib/mercurial.spec
--- a/contrib/mercurial.spec
+++ b/contrib/mercurial.spec
@@ -1,6 +1,23 @@ 
 %global emacs_lispdir %{_datadir}/emacs/site-lisp
+
+%define withpython %{nil}
+
+%if "%{?withpython}"
+
+%global pythonver %{withpython}
+%global pythonname Python-%{withpython}
+%global docutilsname docutils-0.11
+%global pythonhg python-hg
+%global hgpyprefix /usr/%{pythonhg}
+# byte compilation will fail on some some Python /test/ files
+%global _python_bytecompile_errors_terminate_build 0
+
+%else
+
 %global pythonver %(python -c 'import sys;print ".".join(map(str, sys.version_info[:2]))')
 
+%endif
+
 Summary: A fast, lightweight Source Control Management system
 Name: mercurial
 Version: snapshot
@@ -9,11 +26,20 @@  License: GPLv2+
 Group: Development/Tools
 URL: http://mercurial.selenic.com/
 Source0: mercurial-%{version}-%{release}.tar.gz
+%if "%{?withpython}"
+Source1: %{pythonname}.tgz
+Source2: %{docutilsname}.tar.gz
+%endif
 BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root
 
-BuildRequires: python >= 2.4, python-devel, make, gcc, python-docutils >= 0.5, gettext
-Provides: hg = %{version}-%{release}
+BuildRequires: make, gcc, gettext
+%if "%{?withpython}"
+BuildRequires: readline-devel, openssl-devel, ncurses-devel, zlib-devel, bzip2-devel
+%else
+BuildRequires: python >= 2.4, python-devel, python-docutils >= 0.5
 Requires: python >= 2.4
+%endif
+Provides: hg = %{version}-%{release}, mercurial = %{version}-%{release}
 # The hgk extension uses the wish tcl interpreter, but we don't enforce it
 #Requires: tk
 
@@ -22,15 +48,67 @@  Mercurial is a fast, lightweight source 
 for efficient handling of very large distributed projects.
 
 %prep
+
+%if "%{?withpython}"
+%setup -q -n mercurial-%{version}-%{release} -a1 -a2
+# despite the comments in cgi.py, we do this to prevent rpmdeps from picking /usr/local/bin/python up
+sed -i '1c#! /usr/bin/env python' %{pythonname}/Lib/cgi.py
+%else
 %setup -q -n mercurial-%{version}-%{release}
+%endif
 
 %build
+%if "%{?withpython}"
+
+PYPATH=$PWD/%{pythonname}
+cd $PYPATH
+./configure --prefix=%{hgpyprefix}
+make all %{?_smp_mflags}
+cd -
+
+cd %{docutilsname}
+LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py build
+cd -
+
+# verify Python environment
+LD_LIBRARY_PATH=$PYPATH PYTHONPATH=$PWD/%{docutilsname} $PYPATH/python -c 'import sys, zlib, bz2, ssl, curses, readline'
+
+export PATH=$PYPATH:$PATH
+export LD_LIBRARY_PATH=$PYPATH
+export CFLAGS="-L $PYPATH"
+export PYTHONPATH=$PWD/%{docutilsname}
+
+%endif
+
 make all
 
 %install
 rm -rf $RPM_BUILD_ROOT
+
+%if "%{?withpython}"
+
+PYPATH=$PWD/%{pythonname}
+cd $PYPATH
+make install DESTDIR=$RPM_BUILD_ROOT
+# these .a are not necessary and they are readonly and strip fails - kill them!
+rm -f %{buildroot}%{hgpyprefix}/lib/{,python2.*/config}/libpython2.*.a
+cd -
+
+cd %{docutilsname}
+LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py install --root="$RPM_BUILD_ROOT"
+cd -
+
+PATH=$PYPATH:$PATH LD_LIBRARY_PATH=$PYPATH make install DESTDIR=$RPM_BUILD_ROOT PREFIX=%{hgpyprefix} MANDIR=%{_mandir}
+mkdir -p $RPM_BUILD_ROOT%{_bindir}
+( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/hg . )
+( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/python2.? %{pythonhg} )
+
+%else
+
 make install DESTDIR=$RPM_BUILD_ROOT PREFIX=%{_prefix} MANDIR=%{_mandir}
 
+%endif
+
 install -m 755 contrib/hgk $RPM_BUILD_ROOT%{_bindir}/
 install -m 755 contrib/hg-ssh $RPM_BUILD_ROOT%{_bindir}/
 
@@ -71,10 +149,15 @@  rm -rf $RPM_BUILD_ROOT
 %dir %{_sysconfdir}/mercurial
 %dir %{_sysconfdir}/mercurial/hgrc.d
 %config(noreplace) %{_sysconfdir}/mercurial/hgrc.d/mergetools.rc
+%if "%{?withpython}"
+%{_bindir}/%{pythonhg}
+%{hgpyprefix}
+%else
 %if "%{?pythonver}" != "2.4"
 %{_libdir}/python%{pythonver}/site-packages/mercurial-*-py%{pythonver}.egg-info
 %endif
 %{_libdir}/python%{pythonver}/site-packages/mercurial
 %{_libdir}/python%{pythonver}/site-packages/hgext
+%endif
 
 %changelog