Patchwork D9945: debugdiscovery: add flags to run discovery on subsets of the local repo

login
register
mail settings
Submitter phabricator
Date Feb. 2, 2021, 5:44 a.m.
Message ID <differential-rev-PHID-DREV-sdw2ertyrj4kqzvvfbgu-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/48256/
State Superseded
Headers show

Comments

phabricator - Feb. 2, 2021, 5:44 a.m.
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  Generating new repository using strip of local clone is very expensive for large
  repositories. And such large repository are the most likely to requires
  debugging around discovery. So we add a simple way to run discovery using
  provided sets of heads.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D9945

AFFECTED FILES
  mercurial/debugcommands.py
  tests/test-completion.t
  tests/test-setdiscovery.t

CHANGE DETAILS




To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel

Patch

diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t
--- a/tests/test-setdiscovery.t
+++ b/tests/test-setdiscovery.t
@@ -1588,3 +1588,139 @@ 
       common:                    0
       missing:                   1
   common heads: 66f7d451a68b
+
+  $ cd ..
+
+
+Test debuging discovery using different subset of the same repository
+=====================================================================
+
+remote is a local subset
+------------------------
+
+remote will be last 25 heads of the local graph
+
+  $ cd $TESTTMP/manyheads
+  $ hg -R a debugdiscovery \
+  > --debug \
+  > --remote-as-revs 'last(heads(all()), 25)' \
+  > --config devel.discovery.randomize=false
+  query 1; heads
+  searching for changes
+  all remote heads known locally
+  elapsed time:  * seconds (glob)
+  round-trips:                   1
+  heads summary:
+    total common heads:         25
+      also local heads:         25
+      also remote heads:        25
+      both:                     25
+    local heads:               260
+      common:                   25
+      missing:                 235
+    remote heads:               25
+      common:                   25
+      unknown:                   0
+  local changesets:           1340
+    common:                    400
+      heads:                    25
+      roots:                     1
+    missing:                   940
+      heads:                   235
+      roots:                   235
+    first undecided set:       940
+      heads:                   235
+      roots:                   235
+      common:                    0
+      missing:                 940
+  common heads: 0dfd965d91c6 0fe09b60448d 14a17233ce9d 175c0a3072cf 1c51e2c80832 1e51600e0698 24eb5f9bdbab 25ce09526613 36bd00abde57 426989fdefa0 596d87362679 5dd1039ea5c0 5ef24f022278 5f230dc19419 80b39998accb 88f40688ffb5 9e37ddf8c632 abf4d55b075e b2ce801fddfe b368b6ac3ce3 c959bf2e869c c9fba6ba4e2e d783207cf649 d9a51e256f21 e3717a4e3753
+
+local is a local subset
+------------------------
+
+remote will be last 25 heads of the local graph
+
+  $ cd $TESTTMP/manyheads
+  $ hg -R a debugdiscovery b \
+  > --debug \
+  > --local-as-revs 'first(heads(all()), 25)' \
+  > --config devel.discovery.randomize=false
+  comparing with b
+  query 1; heads
+  searching for changes
+  taking quick initial sample
+  query 2; still undecided: 375, sample size is: 81
+  sampling from both directions
+  query 3; still undecided: 3, sample size is: 3
+  3 total queries *s (glob)
+  elapsed time:  * seconds (glob)
+  round-trips:                   3
+  heads summary:
+    total common heads:          1
+      also local heads:          0
+      also remote heads:         0
+      both:                      0
+    local heads:                25
+      common:                    0
+      missing:                  25
+    remote heads:                1
+      common:                    0
+      unknown:                   1
+  local changesets:            400
+    common:                    300
+      heads:                     1
+      roots:                     1
+    missing:                   100
+      heads:                    25
+      roots:                    25
+    first undecided set:       400
+      heads:                    25
+      roots:                     1
+      common:                  300
+      missing:                 100
+  common heads: 3ee37d65064a
+
+both local and remove are subset
+------------------------
+
+remote will be last 25 heads of the local graph
+
+  $ cd $TESTTMP/manyheads
+  $ hg -R a debugdiscovery \
+  > --debug \
+  > --local-as-revs 'first(heads(all()), 25)' \
+  > --remote-as-revs 'last(heads(all()), 25)' \
+  > --config devel.discovery.randomize=false
+  query 1; heads
+  searching for changes
+  taking quick initial sample
+  query 2; still undecided: 375, sample size is: 81
+  sampling from both directions
+  query 3; still undecided: 3, sample size is: 3
+  3 total queries in *s (glob)
+  elapsed time:  * seconds (glob)
+  round-trips:                   3
+  heads summary:
+    total common heads:          1
+      also local heads:          0
+      also remote heads:         0
+      both:                      0
+    local heads:                25
+      common:                    0
+      missing:                  25
+    remote heads:               25
+      common:                    0
+      unknown:                  25
+  local changesets:            400
+    common:                    300
+      heads:                     1
+      roots:                     1
+    missing:                   100
+      heads:                    25
+      roots:                    25
+    first undecided set:       400
+      heads:                    25
+      roots:                     1
+      common:                  300
+      missing:                 100
+  common heads: 3ee37d65064a
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -283,7 +283,7 @@ 
   debugdate: extended
   debugdeltachain: changelog, manifest, dir, template
   debugdirstate: nodates, dates, datesort
-  debugdiscovery: old, nonheads, rev, seed, ssh, remotecmd, insecure
+  debugdiscovery: old, nonheads, rev, seed, local-as-revs, remote-as-revs, ssh, remotecmd, insecure
   debugdownload: output
   debugextensions: template
   debugfileset: rev, all-files, show-matcher, show-stage
diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -69,6 +69,7 @@ 
     pycompat,
     registrar,
     repair,
+    repoview,
     revlog,
     revset,
     revsetlang,
@@ -964,20 +965,73 @@ 
         ),
         (b'', b'rev', [], b'restrict discovery to this set of revs'),
         (b'', b'seed', b'12323', b'specify the random seed use for discovery'),
+        (
+            b'',
+            b'local-as-revs',
+            "",
+            'treat local has having these revisions only',
+        ),
+        (
+            b'',
+            b'remote-as-revs',
+            "",
+            'use local as remote, with only these these revisions',
+        ),
     ]
     + cmdutil.remoteopts,
     _(b'[--rev REV] [OTHER]'),
 )
 def debugdiscovery(ui, repo, remoteurl=b"default", **opts):
-    """runs the changeset discovery protocol in isolation"""
+    """runs the changeset discovery protocol in isolation
+
+    The local peer can be "replaced" by a subset of the local repository by
+    using the `--local-as-revs` flag. Int he same way, usual `remote` peer can
+    be "replaced" by a subset of the local repository using the
+    `--local-as-revs` flag. This is useful to efficiently debug pathological
+    discovery situation.
+    """
     opts = pycompat.byteskwargs(opts)
-    remoteurl, branches = hg.parseurl(ui.expandpath(remoteurl))
-    remote = hg.peer(repo, opts, remoteurl)
-    ui.status(_(b'comparing with %s\n') % util.hidepassword(remoteurl))
+    unfi = repo.unfiltered()
+
+    # setup potential extra filtering
+    local_revs = opts[b"local_as_revs"]
+    remote_revs = opts[b"remote_as_revs"]
 
     # make sure tests are repeatable
     random.seed(int(opts[b'seed']))
 
+    if not remote_revs:
+
+        remoteurl, branches = hg.parseurl(ui.expandpath(remoteurl))
+        remote = hg.peer(repo, opts, remoteurl)
+        ui.status(_(b'comparing with %s\n') % util.hidepassword(remoteurl))
+    else:
+        branches = (None, [])
+        remote_filtered_revs = scmutil.revrange(
+            unfi, [b"not (::(%s))" % remote_revs]
+        )
+        remote_filtered_revs = frozenset(remote_filtered_revs)
+
+        def remote_func(x):
+            return remote_filtered_revs
+
+        repoview.filtertable[b'debug-discovery-remote-filter'] = remote_func
+
+        remote = repo.peer()
+        remote._repo = remote._repo.filtered(b'debug-discovery-remote-filter')
+
+    if local_revs:
+        local_filtered_revs = scmutil.revrange(
+            unfi, [b"not (::(%s))" % local_revs]
+        )
+        local_filtered_revs = frozenset(local_filtered_revs)
+
+        def local_func(x):
+            return local_filtered_revs
+
+        repoview.filtertable[b'debug-discovery-local-filter'] = local_func
+        repo = repo.filtered(b'debug-discovery-local-filter')
+
     data = {}
     if opts.get(b'old'):