Patchwork obsolete: make _computeobsoleteset much faster:

login
register
mail settings
Submitter lc2817@columbia.edu
Date Jan. 14, 2016, 6:09 a.m.
Message ID <bd70fa9f81a0572e3d9e.1452751748@75.1.168.192.in-addr.arpa>
Download mbox | patch
Permalink /patch/12757/
State Accepted
Headers show

Comments

lc2817@columbia.edu - Jan. 14, 2016, 6:09 a.m.
# HG changeset patch
# User Laurent Charignon <lc2817@columbia.edu>
# Date 1452750746 28800
#      Wed Jan 13 21:52:26 2016 -0800
# Node ID bd70fa9f81a0572e3d9e1472b021efe540f4517e
# Parent  443848eece189002c542339dc1cf84f49a94c824
obsolete: make _computeobsoleteset much faster:

This patch makes _computeobsoleteset much faster by looping
over the draft and secrets as opposed to looping over the
successors.
This works because "number of draft and secret" is typically
way smaller(<100) than the number of successor in the repo (~90k in
my checkout of core mercurial as of today). And also because
it is very fast to compute "not public()".

I timed the code with the following setup:
"""
from mercurial import hg, ui, obsolete
ui = ui.ui()
repo = hg.repository(ui, "~/hg")
l = repo.obsstore.successors # This caches the result
"""

With about 90k successors.

k=obsolete._computeobsoleteset(repo) before this patch:
10 loops, best of 3: 33.9 ms per loop

k=obsolete._computeobsoleteset(repo) after this patch:
10000 loops, best of 3: 83.3 µs per loop
Augie Fackler - Jan. 14, 2016, 7:21 p.m.
On Wed, Jan 13, 2016 at 10:09:08PM -0800, lc2817@columbia.edu wrote:
> # HG changeset patch
> # User Laurent Charignon <lc2817@columbia.edu>
> # Date 1452750746 28800
> #      Wed Jan 13 21:52:26 2016 -0800
> # Node ID bd70fa9f81a0572e3d9e1472b021efe540f4517e
> # Parent  443848eece189002c542339dc1cf84f49a94c824
> obsolete: make _computeobsoleteset much faster:

Daaaang. Queued, thanks.

>
> This patch makes _computeobsoleteset much faster by looping
> over the draft and secrets as opposed to looping over the
> successors.
> This works because "number of draft and secret" is typically
> way smaller(<100) than the number of successor in the repo (~90k in
> my checkout of core mercurial as of today). And also because
> it is very fast to compute "not public()".
>
> I timed the code with the following setup:
> """
> from mercurial import hg, ui, obsolete
> ui = ui.ui()
> repo = hg.repository(ui, "~/hg")
> l = repo.obsstore.successors # This caches the result
> """
>
> With about 90k successors.
>
> k=obsolete._computeobsoleteset(repo) before this patch:
> 10 loops, best of 3: 33.9 ms per loop
>
> k=obsolete._computeobsoleteset(repo) after this patch:
> 10000 loops, best of 3: 83.3 µs per loop
>
> diff -r 443848eece18 -r bd70fa9f81a0 mercurial/obsolete.py
> --- a/mercurial/obsolete.py	Wed Jan 13 15:47:37 2016 -0600
> +++ b/mercurial/obsolete.py	Wed Jan 13 21:52:26 2016 -0800
> @@ -1119,12 +1119,11 @@
>  def _computeobsoleteset(repo):
>      """the set of obsolete revisions"""
>      obs = set()
> -    getrev = repo.changelog.nodemap.get
> -    getphase = repo._phasecache.phase
> -    for n in repo.obsstore.successors:
> -        rev = getrev(n)
> -        if rev is not None and getphase(repo, rev):
> -            obs.add(rev)
> +    getnode = repo.changelog.node
> +    notpublic = repo.revs("not public()")
> +    for r in notpublic:
> +        if getnode(r) in repo.obsstore.successors:
> +            obs.add(r)
>      return obs
>
>  @cachefor('unstable')
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> https://selenic.com/mailman/listinfo/mercurial-devel

Patch

diff -r 443848eece18 -r bd70fa9f81a0 mercurial/obsolete.py
--- a/mercurial/obsolete.py	Wed Jan 13 15:47:37 2016 -0600
+++ b/mercurial/obsolete.py	Wed Jan 13 21:52:26 2016 -0800
@@ -1119,12 +1119,11 @@ 
 def _computeobsoleteset(repo):
     """the set of obsolete revisions"""
     obs = set()
-    getrev = repo.changelog.nodemap.get
-    getphase = repo._phasecache.phase
-    for n in repo.obsstore.successors:
-        rev = getrev(n)
-        if rev is not None and getphase(repo, rev):
-            obs.add(rev)
+    getnode = repo.changelog.node
+    notpublic = repo.revs("not public()")
+    for r in notpublic:
+        if getnode(r) in repo.obsstore.successors:
+            obs.add(r)
     return obs
 
 @cachefor('unstable')