Patchwork [6,of,6,foldmap-in-C] dirstate: use parsers.make_file_foldmap when available

login
register
mail settings
Submitter Siddharth Agarwal
Date April 3, 2015, 4:02 a.m.
Message ID <a07e1719e2f8fe9e69f3.1428033739@devbig136.prn2.facebook.com>
Download mbox | patch
Permalink /patch/8473/
State Accepted
Commit 4a4018831d2ebc3c9cae9c6613e6a2497b4f0993
Headers show

Comments

Siddharth Agarwal - April 3, 2015, 4:02 a.m.
# HG changeset patch
# User Siddharth Agarwal <sid0@fb.com>
# Date 1427874273 25200
#      Wed Apr 01 00:44:33 2015 -0700
# Node ID a07e1719e2f8fe9e69f39d839f2e11ba52182c09
# Parent  d6924eb0182c49f916315794f905ed90852a6205
dirstate: use parsers.make_file_foldmap when available

This is a significant performance win on large repositories. perffilefoldmap:

On Linux/gcc, on a test repo with over 500,000 files:
before: wall 0.605021 comb 0.600000 user 0.560000 sys 0.040000 (best of 17)
after:  wall 0.280530 comb 0.280000 user 0.250000 sys 0.030000 (best of 35)

On Mac OS X/clang, on a real-world repo with over 200,000 files:
before: wall 0.281103 comb 0.280000 user 0.260000 sys 0.020000 (best of 34)
after:  wall 0.133622 comb 0.140000 user 0.120000 sys 0.020000 (best of 65)

This visibly impacts status times on case-insensitive file systems. On the Mac
OS X repo, status goes from 3.64 seconds to 3.50.

With the third-party hgwatchman extension [1], 'hg status' on the same repo
goes from 0.80 seconds to 0.65.

[1] https://bitbucket.org/facebook/hgwatchman
Matt Mackall - April 3, 2015, 10:13 p.m.
On Thu, 2015-04-02 at 21:02 -0700, Siddharth Agarwal wrote:
> # HG changeset patch
> # User Siddharth Agarwal <sid0@fb.com>
> # Date 1427874273 25200
> #      Wed Apr 01 00:44:33 2015 -0700
> # Node ID a07e1719e2f8fe9e69f39d839f2e11ba52182c09
> # Parent  d6924eb0182c49f916315794f905ed90852a6205
> dirstate: use parsers.make_file_foldmap when available

These are queued for default, thanks.

Patch

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -88,6 +88,14 @@  class dirstate(object):
 
     @propertycache
     def _filefoldmap(self):
+        try:
+            makefilefoldmap = parsers.make_file_foldmap
+        except AttributeError:
+            pass
+        else:
+            return makefilefoldmap(self._map, util.normcasespec,
+                                   util.normcasefallback)
+
         f = {}
         normcase = util.normcase
         for name, s in self._map.iteritems():