Patchwork [V2] patches: release the GIL while applying the patch

login
register
mail settings
Submitter Boris Feld
Date Feb. 22, 2018, 11:10 a.m.
Message ID <585005c9c4901f4f9484.1519297812@FB>
Download mbox | patch
Permalink /patch/28249/
State Accepted
Headers show

Comments

Boris Feld - Feb. 22, 2018, 11:10 a.m.
# HG changeset patch
# User Boris Feld <boris.feld@octobus.net>
# Date 1517839431 -3600
#      Mon Feb 05 15:03:51 2018 +0100
# Node ID 585005c9c4901f4f94847e8637fbc58cc5b29c56
# Parent  0c34cb461a1ea5d3f8e1300e0b8bc16ed8fa8802
# EXP-Topic parallel-patching
# Available At https://bitbucket.org/octobus/mercurial-devel/
#              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 585005c9c490
patches: release the GIL while applying the patch

This will allow multiple threads to apply patches at the same time.
Yuya Nishihara - Feb. 22, 2018, 12:30 p.m.
On Thu, 22 Feb 2018 12:10:12 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld <boris.feld@octobus.net>
> # Date 1517839431 -3600
> #      Mon Feb 05 15:03:51 2018 +0100
> # Node ID 585005c9c4901f4f94847e8637fbc58cc5b29c56
> # Parent  0c34cb461a1ea5d3f8e1300e0b8bc16ed8fa8802
> # EXP-Topic parallel-patching
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 585005c9c490
> patches: release the GIL while applying the patch
> 
> This will allow multiple threads to apply patches at the same time.

Queued, thanks.
Yuya Nishihara - Feb. 22, 2018, 12:41 p.m.
On Thu, 22 Feb 2018 12:10:12 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld <boris.feld@octobus.net>
> # Date 1517839431 -3600
> #      Mon Feb 05 15:03:51 2018 +0100
> # Node ID 585005c9c4901f4f94847e8637fbc58cc5b29c56
> # Parent  0c34cb461a1ea5d3f8e1300e0b8bc16ed8fa8802
> # EXP-Topic parallel-patching
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 585005c9c490
> patches: release the GIL while applying the patch
> 
> This will allow multiple threads to apply patches at the same time.
> 
> diff --git a/mercurial/cext/mpatch.c b/mercurial/cext/mpatch.c
> --- a/mercurial/cext/mpatch.c
> +++ b/mercurial/cext/mpatch.c
> @@ -109,7 +109,9 @@ static PyObject *patches(PyObject *self,
>  		goto cleanup;
>  	}
>  	out = PyBytes_AsString(result);
> +	Py_BEGIN_ALLOW_THREADS
>  	r = mpatch_apply(out, in, inlen, patch);
> +	Py_END_ALLOW_THREADS

Bad macro. I've inserted one more block to make clang-format happy.

	/* clang-format off */
	{
		Py_BEGIN_ALLOW_THREADS
		r = mpatch_apply(out, in, inlen, patch);
		Py_END_ALLOW_THREADS
	}
	/* clang-format on */
Augie Fackler - Feb. 26, 2018, 4:12 a.m.
On Thu, Feb 22, 2018 at 09:41:01PM +0900, Yuya Nishihara wrote:
> On Thu, 22 Feb 2018 12:10:12 +0100, Boris Feld wrote:
> > # HG changeset patch
> > # User Boris Feld <boris.feld@octobus.net>
> > # Date 1517839431 -3600
> > #      Mon Feb 05 15:03:51 2018 +0100
> > # Node ID 585005c9c4901f4f94847e8637fbc58cc5b29c56
> > # Parent  0c34cb461a1ea5d3f8e1300e0b8bc16ed8fa8802
> > # EXP-Topic parallel-patching
> > # Available At https://bitbucket.org/octobus/mercurial-devel/
> > #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 585005c9c490
> > patches: release the GIL while applying the patch
> >
> > This will allow multiple threads to apply patches at the same time.
> >
> > diff --git a/mercurial/cext/mpatch.c b/mercurial/cext/mpatch.c
> > --- a/mercurial/cext/mpatch.c
> > +++ b/mercurial/cext/mpatch.c
> > @@ -109,7 +109,9 @@ static PyObject *patches(PyObject *self,
> >             goto cleanup;
> >     }
> >     out = PyBytes_AsString(result);
> > +	Py_BEGIN_ALLOW_THREADS
> >     r = mpatch_apply(out, in, inlen, patch);
> > +	Py_END_ALLOW_THREADS
>
> Bad macro. I've inserted one more block to make clang-format happy.

Indeed. Python has a few of these, and I periodically pester some
clang-format devs at Google to try and get some progress. That might
finally have borne fruit in the last month: it sounds like eventually
we'll be able to define a macro to clang-format as ending with one of
{,;}, so the formatter will be able to do less-dumb things.

>
>       /* clang-format off */
>       {
>               Py_BEGIN_ALLOW_THREADS
>               r = mpatch_apply(out, in, inlen, patch);
>               Py_END_ALLOW_THREADS
>       }
>       /* clang-format on */
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Patch

diff --git a/mercurial/cext/mpatch.c b/mercurial/cext/mpatch.c
--- a/mercurial/cext/mpatch.c
+++ b/mercurial/cext/mpatch.c
@@ -109,7 +109,9 @@  static PyObject *patches(PyObject *self,
 		goto cleanup;
 	}
 	out = PyBytes_AsString(result);
+	Py_BEGIN_ALLOW_THREADS
 	r = mpatch_apply(out, in, inlen, patch);
+	Py_END_ALLOW_THREADS
 	if (r < 0) {
 		Py_DECREF(result);
 		result = NULL;