Patchwork [6,of,6,packaging] dockerdeb: rules to build a debian package using docker

login
register
mail settings
Submitter Augie Fackler
Date May 8, 2015, 5:10 p.m.
Message ID <f7c7e1a155383727d54d.1431105009@arthedain.pit.corp.google.com>
Download mbox | patch
Permalink /patch/8976/
State Accepted
Headers show

Comments

Augie Fackler - May 8, 2015, 5:10 p.m.
# HG changeset patch
# User Augie Fackler <augie@google.com>
# Date 1430932539 14400
#      Wed May 06 13:15:39 2015 -0400
# Node ID f7c7e1a155383727d54decd6f31b865b8a52eb8b
# Parent  eb1672b7b846324d4275b49c09d33373e3cb1920
dockerdeb: rules to build a debian package using docker

Currently only supports jessie (current stable), but other version
should be trivial.
Gregory Szorc - May 8, 2015, 6:04 p.m.
On Fri, May 8, 2015 at 10:10 AM, Augie Fackler <raf@durin42.com> wrote:

> # HG changeset patch
> # User Augie Fackler <augie@google.com>
> # Date 1430932539 14400
> #      Wed May 06 13:15:39 2015 -0400
> # Node ID f7c7e1a155383727d54decd6f31b865b8a52eb8b
> # Parent  eb1672b7b846324d4275b49c09d33373e3cb1920
> dockerdeb: rules to build a debian package using docker
>
> Currently only supports jessie (current stable), but other version
> should be trivial.
>
> diff --git a/Makefile b/Makefile
> --- a/Makefile
> +++ b/Makefile
> @@ -163,6 +163,10 @@ debian-jessie:
>         mv debbuild/*.deb packages/debian-jessie
>         rm -rf debbuild
>
> +docker-debian-jessie:
> +       mkdir -p packages/debian/jessie
> +       contrib/dockerdeb jessie
> +
>  fedora20:
>         mkdir -p packages/fedora20
>         contrib/buildrpm
> diff --git a/contrib/docker/debian-jessie b/contrib/docker/debian-jessie
> new file mode 100644
> --- /dev/null
> +++ b/contrib/docker/debian-jessie
> @@ -0,0 +1,11 @@
> +FROM debian:jessie
> +RUN apt-get update && apt-get install -y \
> +  build-essential \
> +  debhelper \
> +  dh-python \
> +  devscripts \
> +  python \
> +  python-all-dev \
> +  python-docutils \
> +  zip \
> +  unzip
> diff --git a/contrib/dockerdeb b/contrib/dockerdeb
> new file mode 100755
> --- /dev/null
> +++ b/contrib/dockerdeb
> @@ -0,0 +1,39 @@
> +#!/bin/bash -eu
> +
> +. $(dirname $0)/dockerlib.sh
> +. $(dirname $0)/packagelib.sh
> +
> +BUILDDIR=$(dirname $0)
> +export ROOTDIR=$(cd $BUILDDIR/..; pwd)
> +
> +checkdocker
> +
> +PLATFORM="debian-$1"
> +shift # extra params are passed to build process
> +
> +initcontainer $PLATFORM
> +
> +DEBBUILDDIR=$ROOTDIR/packages/$PLATFORM
> +contrib/builddeb --debbuilddir $DEBBUILDDIR/staged --prepare
> +
> +DSHARED=/mnt/shared/
> +if [ $(uname) = "Darwin" ] ; then
> +    $DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED -v
> $PWD:/mnt/hg $CONTAINER \
> +            sh -c "cd /mnt/hg && make clean && make local"
> +fi
> +$DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED -v $PWD:/mnt/hg
> $CONTAINER \
> +  sh -c "cd /mnt/hg && make PREFIX=$DSHARED/staged/usr install"
> +$DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED $CONTAINER \
> +  dpkg-deb --build $DSHARED/staged
> +if [ $(uname) = "Darwin" ] ; then
> +    $DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED -v
> $PWD:/mnt/hg $CONTAINER \
> +            sh -c "cd /mnt/hg && make clean"
> +fi
>

As you alluded to in an earlier patch, uids are fragile. And mounting
volumes will almost certainly result in uid/gid badness, especially when
host OSs differ.

When I need to transfer files from my host to Docker, I typically end up
going through an intermediate, such as a tar file (with normalized
permissions) or even a Mercurial bundle so this mismatch doesn't occur.
This patch is probably fine for now. But if my experience is an indication,
this approach is brittle and will need to be replaced by something more
complicated :/
Augie Fackler - May 8, 2015, 6:07 p.m.
> On May 8, 2015, at 2:04 PM, Gregory Szorc <gregory.szorc@gmail.com> wrote:
> 
> On Fri, May 8, 2015 at 10:10 AM, Augie Fackler <raf@durin42.com> wrote:
> # HG changeset patch
> # User Augie Fackler <augie@google.com>
> # Date 1430932539 14400
> #      Wed May 06 13:15:39 2015 -0400
> # Node ID f7c7e1a155383727d54decd6f31b865b8a52eb8b
> # Parent  eb1672b7b846324d4275b49c09d33373e3cb1920
> dockerdeb: rules to build a debian package using docker
> 
> Currently only supports jessie (current stable), but other version
> should be trivial.
> 
> 

[...]

> +$DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED -v $PWD:/mnt/hg $CONTAINER \
> +  sh -c "cd /mnt/hg && make PREFIX=$DSHARED/staged/usr install"
> +$DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED $CONTAINER \
> +  dpkg-deb --build $DSHARED/staged
> +if [ $(uname) = "Darwin" ] ; then
> +    $DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED -v $PWD:/mnt/hg $CONTAINER \
> +            sh -c "cd /mnt/hg && make clean"
> +fi
> 
> As you alluded to in an earlier patch, uids are fragile. And mounting volumes will almost certainly result in uid/gid badness, especially when host OSs differ.
> 
> When I need to transfer files from my host to Docker, I typically end up going through an intermediate, such as a tar file (with normalized permissions) or even a Mercurial bundle so this mismatch doesn't occur. This patch is probably fine for now. But if my experience is an indication, this approach is brittle and will need to be replaced by something more complicated :/

Yup. This is super breaky, but the way we construct containers is mostly-robust in the face of these problems, which surprises me more than a little. The one patch to make things work with boot2docker feels hopelessly dirty, but on the other hand I was able to build debs and rpms on my Mac reliably, so it seems to be fine for now.

The "good" news is that when we do fix things, we now have a good place to put that icky code.
Pierre-Yves David - May 8, 2015, 9:37 p.m.
On 05/08/2015 10:10 AM, Augie Fackler wrote:
> # HG changeset patch
> # User Augie Fackler <augie@google.com>
> # Date 1430932539 14400
> #      Wed May 06 13:15:39 2015 -0400
> # Node ID f7c7e1a155383727d54decd6f31b865b8a52eb8b
> # Parent  eb1672b7b846324d4275b49c09d33373e3cb1920
> dockerdeb: rules to build a debian package using docker
>
> Currently only supports jessie (current stable), but other version
> should be trivial.

Okay, these patch are pushed to the clowcopter. Much dirtyness, but one 
as to start somewhere.

Patch

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -163,6 +163,10 @@  debian-jessie:
 	mv debbuild/*.deb packages/debian-jessie
 	rm -rf debbuild
 
+docker-debian-jessie:
+	mkdir -p packages/debian/jessie
+	contrib/dockerdeb jessie
+
 fedora20:
 	mkdir -p packages/fedora20
 	contrib/buildrpm
diff --git a/contrib/docker/debian-jessie b/contrib/docker/debian-jessie
new file mode 100644
--- /dev/null
+++ b/contrib/docker/debian-jessie
@@ -0,0 +1,11 @@ 
+FROM debian:jessie
+RUN apt-get update && apt-get install -y \
+  build-essential \
+  debhelper \
+  dh-python \
+  devscripts \
+  python \
+  python-all-dev \
+  python-docutils \
+  zip \
+  unzip
diff --git a/contrib/dockerdeb b/contrib/dockerdeb
new file mode 100755
--- /dev/null
+++ b/contrib/dockerdeb
@@ -0,0 +1,39 @@ 
+#!/bin/bash -eu
+
+. $(dirname $0)/dockerlib.sh
+. $(dirname $0)/packagelib.sh
+
+BUILDDIR=$(dirname $0)
+export ROOTDIR=$(cd $BUILDDIR/..; pwd)
+
+checkdocker
+
+PLATFORM="debian-$1"
+shift # extra params are passed to build process
+
+initcontainer $PLATFORM
+
+DEBBUILDDIR=$ROOTDIR/packages/$PLATFORM
+contrib/builddeb --debbuilddir $DEBBUILDDIR/staged --prepare
+
+DSHARED=/mnt/shared/
+if [ $(uname) = "Darwin" ] ; then
+    $DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED -v $PWD:/mnt/hg $CONTAINER \
+            sh -c "cd /mnt/hg && make clean && make local"
+fi
+$DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED -v $PWD:/mnt/hg $CONTAINER \
+  sh -c "cd /mnt/hg && make PREFIX=$DSHARED/staged/usr install"
+$DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED $CONTAINER \
+  dpkg-deb --build $DSHARED/staged
+if [ $(uname) = "Darwin" ] ; then
+    $DOCKER run -u $DBUILDUSER --rm -v $DEBBUILDDIR:$DSHARED -v $PWD:/mnt/hg $CONTAINER \
+            sh -c "cd /mnt/hg && make clean"
+fi
+
+gethgversion
+
+rm -r $DEBBUILDDIR/staged
+mv $DEBBUILDDIR/staged.deb $DEBBUILDDIR/mercurial-$version-$release.deb
+
+echo
+echo "Build complete - results can be found in $DEBBUILDDIR"