Patchwork [STABLE] changegroup: store old heads as a set

login
register
mail settings
Submitter Gregory Szorc
Date March 24, 2017, 2:59 a.m.
Message ID <ba2c20c45b3e0202e52b.1490324340@ubuntu-vm-main>
Download mbox | patch
Permalink /patch/19616/
State Accepted
Headers show

Comments

Gregory Szorc - March 24, 2017, 2:59 a.m.
# HG changeset patch
# User Gregory Szorc <gregory.szorc@gmail.com>
# Date 1490324099 25200
#      Thu Mar 23 19:54:59 2017 -0700
# Branch stable
# Node ID ba2c20c45b3e0202e52bbd3252f9a8214d8a2e06
# Parent  2915cc1d3429f9b11c9cf48d45990f982fff8e7a
changegroup: store old heads as a set

Previously, the "oldheads" variable was a list. On a repository at
Mozilla with 46,492 heads, profiling revealed that list membership
testing was dominating execution time of applying small changegroups.

This patch converts the list of old heads to a set. This makes
membership testing significantly faster. On the aforementioned
repository with 46,492 heads:

$ hg unbundle <file with 1 changeset>
before: 18.535s wall
after:   1.303s

Consumers of this variable only check for truthiness (`if oldheads`),
length (`len(oldheads)`), and (most importantly) item membership
(`h not in oldheads` - which occurs twice). So, the change to a set
should be safe and suitable for stable.

The practical effect of this change is that changegroup application
and related operations (like `hg push`) no longer exhibit an O(n^2)
CPU explosion as the number of heads grows.
Augie Fackler - March 24, 2017, 3:17 a.m.
> On Mar 23, 2017, at 10:59 PM, Gregory Szorc <gregory.szorc@gmail.com> wrote:
> 
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1490324099 25200
> #      Thu Mar 23 19:54:59 2017 -0700
> # Branch stable
> # Node ID ba2c20c45b3e0202e52bbd3252f9a8214d8a2e06
> # Parent  2915cc1d3429f9b11c9cf48d45990f982fff8e7a
> changegroup: store old heads as a set

Queued for stable. Thanks.


> Previously, the "oldheads" variable was a list. On a repository at
> Mozilla with 46,492 heads, profiling revealed that list membership
> testing was dominating execution time of applying small changegroups.
> 
> This patch converts the list of old heads to a set. This makes
> membership testing significantly faster. On the aforementioned
> repository with 46,492 heads:
> 
> $ hg unbundle <file with 1 changeset>
> before: 18.535s wall
> after:   1.303s
> 
> Consumers of this variable only check for truthiness (`if oldheads`),
> length (`len(oldheads)`), and (most importantly) item membership
> (`h not in oldheads` - which occurs twice). So, the change to a set
> should be safe and suitable for stable.
> 
> The practical effect of this change is that changegroup application
> and related operations (like `hg push`) no longer exhibit an O(n^2)
> CPU explosion as the number of heads grows.
> 
> diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
> --- a/mercurial/changegroup.py
> +++ b/mercurial/changegroup.py
> @@ -293,7 +293,7 @@ class cg1unpacker(object):
>                 # will not see an inconsistent view
>                 cl = repo.changelog
>                 cl.delayupdate(tr)
> -                oldheads = cl.heads()
> +                oldheads = set(cl.heads())
> 
>                 trp = weakref.proxy(tr)
>                 # pull off the changeset group
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Patch

diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -293,7 +293,7 @@  class cg1unpacker(object):
                 # will not see an inconsistent view
                 cl = repo.changelog
                 cl.delayupdate(tr)
-                oldheads = cl.heads()
+                oldheads = set(cl.heads())
 
                 trp = weakref.proxy(tr)
                 # pull off the changeset group