Patchwork [4,of,4,V3] dirstate: add test for non-normal map consistency

login
register
mail settings
Submitter Laurent Charignon
Date Dec. 17, 2015, 6:24 p.m.
Message ID <9eddea0f60cd7b7354af.1450376686@lcharignon-mbp.local>
Download mbox | patch
Permalink /patch/12108/
State Changes Requested, archived
Headers show

Comments

Laurent Charignon - Dec. 17, 2015, 6:24 p.m.
# HG changeset patch
# User Laurent Charignon <lcharignon@fb.com>
# Date 1450376462 28800
#      Thu Dec 17 10:21:02 2015 -0800
# Node ID 9eddea0f60cd7b7354aff2166e94dd6ee39ecacc
# Parent  530d455d2eb83e6b70c3887f5998b84681b5783e
dirstate: add test for non-normal map consistency

This adds a test extension to check that the non-normal map contains the
expected entries. It wraps several methods of the dirstate to check that
the non-normal map has the correct values before and after the call. The
extension lives in contrib so that paranoid developers can easily
enable it to make sure that the non-normal map is consistent across more
complex operations than the included tests.

I tested the whole series with hgwatchman against mozilla central with the perf
contrib extensions. For this test I had to modify hgwatchman to use the
non-normal map.

These are the results for hg perfstatus (86x improvement):

With the hgwatchman change and non-normal map
! wall 0.007389 comb 0.000000 user 0.000000 sys 0.000000 (best of 256)

Without the non-normal map
! wall 0.642929 comb 0.640000 user 0.380000 sys 0.260000 (best of 3)

With the changes, on mozilla-central hg status goes from 0.7s to 0.2s on my
laptop.

Patch

diff --git a/contrib/dirstatenonnormalcheck.py b/contrib/dirstatenonnormalcheck.py
new file mode 100644
--- /dev/null
+++ b/contrib/dirstatenonnormalcheck.py
@@ -0,0 +1,60 @@ 
+# dirstatenonnormalcheck.py - extension to check the consistency of the
+# dirstate's non-normal map
+#
+# For most operations on dirstate, this extensions checks that the nonnormalmap
+# contains the right entries.
+# It compares the nonnormal file to a nonnormalmap built from the map of all
+# the files in the dirstate to check that they contain the same files.
+
+from __future__ import absolute_import
+
+from mercurial import (
+    dirstate,
+    extensions,
+)
+
+def nonnormalentries(dmap):
+    """Compute nonnormal entries from dirstate's dmap"""
+    res = {}
+    for f, e in dmap.iteritems():
+        if e[0] != 'n' or e[3] == -1:
+            res[f] = tuple(e)
+    return res
+
+def checkconsistency(ui, orig, dmap, _nonnormalmap, label):
+    """Compute nonnormamap from dmap, check that it matches nonnormalmap"""
+    nonnormalcomputedmap = nonnormalentries(dmap)
+    nonnormalmap = {}
+    for k in _nonnormalmap:
+        nonnormalmap[k] = tuple(_nonnormalmap[k])
+    if nonnormalmap != nonnormalcomputedmap:
+        ui.develwarn("%s call to %s\n" % (label, orig))
+        ui.develwarn("inconsistency in nonnormalmap\n")
+        ui.develwarn("[nonnormalmap] %s\n" % nonnormalmap)
+        ui.develwarn("[map] %s\n" % nonnormalcomputedmap)
+
+def _checkdirstate(orig, self, arg):
+    """Check nonnormal map consistency before and after the call to orig"""
+    checkconsistency(self._ui, orig, self._map, self._nonnormalmap, "before")
+    r =  orig(self, arg)
+    checkconsistency(self._ui, orig, self._map, self._nonnormalmap, "after")
+    return r
+
+def extsetup(ui):
+    """Wrap functions modifying dirstate to check nonnormalmap consistency"""
+    dirstatecl = dirstate.dirstate
+    devel = ui.configbool('devel', 'all-warnings')
+    paranoid = ui.configbool('experimental', 'nonnormalparanoidcheck')
+    if devel:
+        extensions.wrapfunction(dirstatecl, '_writedirstate', _checkdirstate)
+        if paranoid:
+            # We don't do all these checks when paranoid is disable as it would
+            # make the extension run very slowly on large repos
+            extensions.wrapfunction(dirstatecl, 'normallookup', _checkdirstate)
+            extensions.wrapfunction(dirstatecl, 'otherparent', _checkdirstate)
+            extensions.wrapfunction(dirstatecl, 'normal', _checkdirstate)
+            extensions.wrapfunction(dirstatecl, 'write', _checkdirstate)
+            extensions.wrapfunction(dirstatecl, 'add', _checkdirstate)
+            extensions.wrapfunction(dirstatecl, 'remove', _checkdirstate)
+            extensions.wrapfunction(dirstatecl, 'merge', _checkdirstate)
+            extensions.wrapfunction(dirstatecl, 'drop', _checkdirstate)
diff --git a/tests/test-dirstate-nonnormalmap.t b/tests/test-dirstate-nonnormalmap.t
new file mode 100644
--- /dev/null
+++ b/tests/test-dirstate-nonnormalmap.t
@@ -0,0 +1,22 @@ 
+  $ cat >> $HGRCPATH << EOF
+  > [ui]
+  > logtemplate="{rev}:{node|short} ({phase}) [{tags} {bookmarks}] {desc|firstline}\n"
+  > [extensions]
+  > dirstateparanoidcheck = $TESTDIR/../contrib/dirstatenonnormalcheck.py
+  > [experimental]
+  > nonnormalparanoidcheck = True
+  > [devel]
+  > all-warnings=True
+  > EOF
+  $ mkcommit() {
+  >    echo "$1" > "$1"
+  >    hg add "$1"
+  >    hg ci -m "add $1"
+  > }
+
+  $ hg init testrepo
+  $ cd testrepo
+  $ mkcommit a
+  $ mkcommit b
+  $ mkcommit c
+  $ hg status