Patchwork [7,of,7,V6] dirstate: call the C implementation of nonnonormalentries when available

login
register
mail settings
Submitter Laurent Charignon
Date Dec. 22, 2015, 12:33 a.m.
Message ID <ae3e76005e5fb179da5e.1450744396@mbuchanan-mbp.DHCP.thefacebook.com>
Download mbox | patch
Permalink /patch/12219/
State Superseded, archived
Commit bc97b9af4e629ce28613c91f8fd96ba4721eab67
Headers show

Comments

Laurent Charignon - Dec. 22, 2015, 12:33 a.m.
# HG changeset patch
# User Laurent Charignon <lcharignon@fb.com>
# Date 1450744054 28800
#      Mon Dec 21 16:27:34 2015 -0800
# Node ID ae3e76005e5fb179da5e0932a3c067cf166bf543
# Parent  cc3b2338b18a45db45a2dcab757455c63e6de0d4
dirstate: call the C implementation of nonnonormalentries when available

Before this patch, we were using python code for computing the nonnormal
dirstate entries. This patch makes us use the C implementation of the function
when it is available.

Using the nonnormal set in hgwatchman improves hg status performance. Below
are the numbers for mozilla-central.

with the changes:
$ hg perfstatus
    ! wall 0.010632 comb 0.000000 user 0.000000 sys 0.000000 (best of 246)

without the changes:
$ hg perfstatus
    ! wall 0.036442 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)

On mozilla-central the improvement to hg status is ~20%, on our big repos, the
win is ~40%.
Martin von Zweigbergk - Dec. 22, 2015, 4:41 p.m.
On Mon, Dec 21, 2015 at 4:39 PM Laurent Charignon <lcharignon@fb.com> wrote:

> # HG changeset patch
> # User Laurent Charignon <lcharignon@fb.com>
> # Date 1450744054 28800
> #      Mon Dec 21 16:27:34 2015 -0800
> # Node ID ae3e76005e5fb179da5e0932a3c067cf166bf543
> # Parent  cc3b2338b18a45db45a2dcab757455c63e6de0d4
> dirstate: call the C implementation of nonnonormalentries when available
>
> Before this patch, we were using python code for computing the nonnormal
> dirstate entries. This patch makes us use the C implementation of the
> function
> when it is available.
>
> Using the nonnormal set in hgwatchman improves hg status performance. Below
> are the numbers for mozilla-central.
>
> with the changes:
> $ hg perfstatus
>     ! wall 0.010632 comb 0.000000 user 0.000000 sys 0.000000 (best of 246)
>
> without the changes:
> $ hg perfstatus
>     ! wall 0.036442 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)
>
> On mozilla-central the improvement to hg status is ~20%, on our big repos,
> the
> win is ~40%.
>

Actually, I think my question about what this means in practice may remain.
Does perfstatus re-read distate on each iteration or does it just iterate
over the already-read _nonnormalentries? Under what circumstances would
that happen in practice? I'm guessing plain "hg status" will read it only
once, but maybe some other commands will. I know you're working on this
because you're fixing a real problem, so I trust that it really is fixing
the problem, but I'd appreciate to see some of those numbers here. Or
perhaps I'm just missing some context here. I have not been following
revisions 2-5 of this series very closely.

Patch

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -38,8 +38,11 @@  def _getfsnow(vfs):
 
 def nonnomalentries(dmap):
     '''Compute the nonnormal dirstate entries from the dmap'''
-    return set(fname for fname, e in dmap.iteritems()
-           if e[0] != 'n' or e[3] == -1)
+    try:
+        return parsers.nonnormalentries(dmap)
+    except AttributeError:
+        return set(fname for fname, e in dmap.iteritems()
+                   if e[0] != 'n' or e[3] == -1)
 
 def _trypending(root, vfs, filename):
     '''Open  file to be read according to HG_PENDING environment variable