Patchwork D262: test-rebase: add a brute force test

login
register
mail settings
Submitter phabricator
Date Aug. 7, 2017, 4:11 p.m.
Message ID <differential-rev-PHID-DREV-cwhqiebh7fnqgohbpqiy-req@phab.mercurial-scm.org>
Download mbox | patch
Permalink /patch/22733/
State Superseded
Headers show

Comments

phabricator - Aug. 7, 2017, 4:11 p.m.
quark created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Rebase is becoming more complex and it looks like a good idea to try some
  brute force enumeration to cover cases that are hard to find manually.
  
  Using brute force to generate repos in different shapes and enumerating the
  rebase source and destination would generate too many cases that takes too
  long to compute. This patch limits the "brute force" to only the "rebase
  source" part. Repo and destination are still manual.
  
  The added test cases are crafted manually to reveal some behaviors that are
  not covered by other tests:
  
  - "revlog index out of range" crash
  - after rebase, p1 == p2, p2 != null
  - "nothing to merge" abort
  
  In the future we might want to add more tests here. For now I'm more
  interested in revealing interesting behaviors in a minified way. I tried
  some more complex cases but didn't find other interesting behaviors.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D262

AFFECTED FILES
  tests/bruterebase.py
  tests/test-rebase-brute-force.t

CHANGE DETAILS




To: quark, #hg-reviewers
Cc: mercurial-devel
phabricator - Aug. 7, 2017, 4:59 p.m.
lothiraldan added a comment.


  Thanks for the patch, more testing is always good.
  
  I'm wondering if we could use hypothesis (https://github.com/HypothesisWorks/hypothesis-python) for more tests like that. One article I found interesting was: http://hypothesis.works/articles/rule-based-stateful-testing/

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D262

To: quark, #hg-reviewers
Cc: lothiraldan, mercurial-devel
phabricator - Aug. 7, 2017, 5:07 p.m.
quark added a comment.


  While I think hypothesis might be somehow useful. It is to generate random test cases, which is different from the efficient brute force here.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D262

To: quark, #hg-reviewers
Cc: lothiraldan, mercurial-devel

Patch

diff --git a/tests/test-rebase-brute-force.t b/tests/test-rebase-brute-force.t
new file mode 100644
--- /dev/null
+++ b/tests/test-rebase-brute-force.t
@@ -0,0 +1,55 @@ 
+  $ cat >> $HGRCPATH <<EOF
+  > [extensions]
+  > drawdag=$TESTDIR/drawdag.py
+  > bruterebase=$TESTDIR/bruterebase.py
+  > [experimental]
+  > evolution=createmarkers,allowunstable
+  > EOF
+  $ init() {
+  >   N=`expr ${N:-0} + 1`
+  >   cd $TESTTMP && hg init repo$N && cd repo$N
+  >   hg debugdrawdag
+  > }
+
+Source looks like "N"
+
+  $ init <<'EOS'
+  > C D
+  > |\|
+  > A B Z
+  > EOS
+
+  $ hg debugbruterebase 'all()-Z' Z
+     A: A':Z
+     B: B':Z
+    AB: A':Z B':Z
+     C: ABORT: cannot use revision 3 as base, result would have 3 parents
+    AC: A':Z C':A'B
+    BC: B':Z C':B'A
+   ABC: A':Z B':Z C':A'B'
+     D: D':Z
+    AD: A':Z D':Z
+    BD: B':Z D':B'
+   ABD: A':Z B':Z D':B'
+    CD: CRASH: revlog index out of range
+   ACD: A':Z C':A'A' D':Z
+   BCD: B':Z C':B'A D':B'
+  ABCD: A':Z B':Z C':A'B' D':B'
+
+Moving backwards
+
+  $ init <<'EOS'
+  > C
+  > |\
+  > A B
+  > |
+  > Z
+  > EOS
+  $ hg debugbruterebase 'all()-Z' Z
+    B: B':Z
+    A: 
+   BA: B':Z
+    C: ABORT: cannot use revision 3 as base, result would have 3 parents
+   BC: B':Z C':B'A
+   AC: 
+  BAC: ABORT: nothing to merge
diff --git a/tests/bruterebase.py b/tests/bruterebase.py
new file mode 100644
--- /dev/null
+++ b/tests/bruterebase.py
@@ -0,0 +1,69 @@ 
+# bruterebase.py - brute force rebase testing
+#
+# Copyright 2017 Facebook, Inc.
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import
+
+from mercurial import (
+    error,
+    registrar,
+    revsetlang,
+)
+
+from hgext import rebase
+
+cmdtable = {}
+command = registrar.command(cmdtable)
+
+@command('debugbruterebase')
+def debugbruterebase(ui, repo, source, dest):
+    """for every non-empty subset of source, run rebase -r subset -d dest
+
+    Print one line summary for each subset. Assume obsstore is enabled.
+    """
+    srevs = list(repo.revs(source))
+
+    with repo.wlock(), repo.lock():
+        repolen = len(repo)
+        cl = repo.changelog
+
+        def getdesc(rev):
+            result = cl.changelogrevision(rev).description
+            if rev >= repolen:
+                result += "'"
+            return result
+
+        for i in xrange(1, 2 ** len(srevs)):
+            subset = [rev for j, rev in enumerate(srevs) if i & (1 << j) != 0]
+            spec = revsetlang.formatspec('%ld', subset)
+            tr = repo.transaction('rebase')
+            tr.report = lambda x: 0 # hide "transaction abort"
+
+            ui.pushbuffer()
+            try:
+                rebase.rebase(ui, repo, dest=dest, rev=[spec])
+            except error.Abort as ex:
+                summary = 'ABORT: %s' % ex
+            except Exception as ex:
+                summary = 'CRASH: %s' % ex
+            else:
+                # short summary about new nodes
+                cl = repo.changelog
+                descs = []
+                for rev in xrange(repolen, len(repo)):
+                    desc = '%s:' % getdesc(rev)
+                    for prev in cl.parentrevs(rev):
+                        if prev > -1:
+                            desc += getdesc(prev)
+                    descs.append(desc)
+                descs.sort()
+                summary = ' '.join(descs)
+            ui.popbuffer()
+            repo.vfs.tryunlink('rebasestate')
+
+            subsetdesc = ''.join(getdesc(rev) for rev in subset)
+            ui.write(('%s: %s\n') % (subsetdesc.rjust(len(srevs)), summary))
+            tr.abort()