Patchwork exchange: allow fallbackheads to use lazy set behavior

login
register
mail settings
Submitter Durham Goode
Date Sept. 8, 2015, 12:13 a.m.
Message ID <c595696df8614f58194e.1441671189@dev2000.prn2.facebook.com>
Download mbox | patch
Permalink /patch/10415/
State Accepted
Delegated to: Augie Fackler
Headers show

Comments

Durham Goode - Sept. 8, 2015, 12:13 a.m.
# HG changeset patch
# User Durham Goode <durham@fb.com>
# Date 1441670915 25200
#      Mon Sep 07 17:08:35 2015 -0700
# Node ID c595696df8614f58194e210f8b3b1d4c1a25926e
# Parent  d72ff051c6c6dee945690b0e5554a0450820e93e
exchange: allow fallbackheads to use lazy set behavior

The common ancestor set implementation was made lazy a couple years ago, but
this piece of code still required processing the entire repo by putting set()
around the lazy set. The code was introduced in 5653f2d166ea, a year before the
lazy ancestor set was added.

Dropping the set() shaves 3.5 seconds off of 'push -r' in repos with hundreds of
thousands of commits.
Augie Fackler - Sept. 8, 2015, 5:01 p.m.
On Mon, Sep 07, 2015 at 05:13:09PM -0700, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1441670915 25200
> #      Mon Sep 07 17:08:35 2015 -0700
> # Node ID c595696df8614f58194e210f8b3b1d4c1a25926e
> # Parent  d72ff051c6c6dee945690b0e5554a0450820e93e
> exchange: allow fallbackheads to use lazy set behavior

Queued, awesome.

>
> The common ancestor set implementation was made lazy a couple years ago, but
> this piece of code still required processing the entire repo by putting set()
> around the lazy set. The code was introduced in 5653f2d166ea, a year before the
> lazy ancestor set was added.
>
> Dropping the set() shaves 3.5 seconds off of 'push -r' in repos with hundreds of
> thousands of commits.
>
> diff --git a/mercurial/exchange.py b/mercurial/exchange.py
> --- a/mercurial/exchange.py
> +++ b/mercurial/exchange.py
> @@ -147,7 +147,7 @@ class pushoperation(object):
>          #
>          # We can pick:
>          # * missingheads part of common (::commonheads)
> -        common = set(self.outgoing.common)
> +        common = self.outgoing.common
>          nm = self.repo.changelog.nodemap
>          cheads = [node for node in self.revs if nm[node] in common]
>          # and
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> https://selenic.com/mailman/listinfo/mercurial-devel
Pierre-Yves David - Sept. 8, 2015, 11 p.m.
On 09/07/2015 05:13 PM, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1441670915 25200
> #      Mon Sep 07 17:08:35 2015 -0700
> # Node ID c595696df8614f58194e210f8b3b1d4c1a25926e
> # Parent  d72ff051c6c6dee945690b0e5554a0450820e93e
> exchange: allow fallbackheads to use lazy set behavior
>
> The common ancestor set implementation was made lazy a couple years ago, but
> this piece of code still required processing the entire repo by putting set()
> around the lazy set. The code was introduced in 5653f2d166ea, a year before the
> lazy ancestor set was added.
>
> Dropping the set() shaves 3.5 seconds off of 'push -r' in repos with hundreds of
> thousands of commits.

Nice. out of curiousity. 3.5 out of how much?
Durham Goode - Sept. 9, 2015, 1 a.m.
On 9/8/15, 4:00 PM, "Pierre-Yves David" <pierre-yves.david@ens-lyon.org> wrote:

>

>

>On 09/07/2015 05:13 PM, Durham Goode wrote:

>> # HG changeset patch

>> # User Durham Goode <durham@fb.com>

>> # Date 1441670915 25200

>> #      Mon Sep 07 17:08:35 2015 -0700

>> # Node ID c595696df8614f58194e210f8b3b1d4c1a25926e

>> # Parent  d72ff051c6c6dee945690b0e5554a0450820e93e

>> exchange: allow fallbackheads to use lazy set behavior

>>

>> The common ancestor set implementation was made lazy a couple years ago, but

>> this piece of code still required processing the entire repo by putting set()

>> around the lazy set. The code was introduced in 5653f2d166ea, a year before the

>> lazy ancestor set was added.

>>

>> Dropping the set() shaves 3.5 seconds off of 'push -r' in repos with hundreds of

>> thousands of commits.

>

>Nice. out of curiousity. 3.5 out of how much?


The client side logic goes from 4.1s to 0.6s.  The server side logic takes 7-10s, so the overall user facing improvement is notable, but not massive.  But now we don't have to worry about the client anymore.

Note: this was with the pushrebase extension enabled, which disables discovery.  Without it enabled, the discovery operation is actually quite expensive (6s or so). I didn't investigate why it was slow.

Patch

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -147,7 +147,7 @@  class pushoperation(object):
         #
         # We can pick:
         # * missingheads part of common (::commonheads)
-        common = set(self.outgoing.common)
+        common = self.outgoing.common
         nm = self.repo.changelog.nodemap
         cheads = [node for node in self.revs if nm[node] in common]
         # and