Patchwork [9,of,9] verify: show progress while verifying dirlogs

login
register
mail settings
Submitter Martin von Zweigbergk
Date Feb. 12, 2016, 10:42 p.m.
Message ID <CAESOdVDFx-ZKqjs=p6E+u0Pxmpfw7jy6b6CrXQkXw++R5+JZHw@mail.gmail.com>
Download mbox | patch
Permalink /patch/13163/
State Superseded
Commit 53f42c8d5f71c3f5caf11a00f827a47cea756645
Headers show

Comments

Martin von Zweigbergk - Feb. 12, 2016, 10:42 p.m.
# HG changeset patch
# User Martin von Zweigbergk <martinvonz@google.com>
# Date 1455233936 28800
#      Thu Feb 11 15:38:56 2016 -0800
# Node ID 2671d0d981316b1fa36706391044c3da80a0bf9d
# Parent  9d77de5a51f6baeb6388ae4ddd064a3c130d5758
verify: show progress while verifying dirlogs

In repos with treemanifests, the non-root-directory dirlogs often have
many more total revisions than the root manifest log has. This change
adds progress out to that part of 'hg verify'. Since the verification
is recursive along the directory tree, we don't know how many total
revisions there are at the beginning of the command, so instead we
report progress in units of directories, much like we report progress
for verification of files today.

I'm not very happy with passing both 'storefiles' and 'progress' into
the recursive calls. I tried passing just a 'visitdir(dir)' callback
in, but the results did not seem better overall. I'm happy to update
if anyone has better ideas.

                 self.warn(_("warning: orphan revlog '%s'") % f)

Patch

diff --git a/mercurial/verify.py b/mercurial/verify.py
--- a/mercurial/verify.py
+++ b/mercurial/verify.py
@@ -197,7 +197,8 @@ 
         ui.progress(_('checking'), None)
         return mflinkrevs, filelinkrevs

-    def _verifymanifest(self, mflinkrevs, dir="", storefiles=None):
+    def _verifymanifest(self, mflinkrevs, dir="", storefiles=None,
+                        progress=None):
         repo = self.repo
         ui = self.ui
         mf = self.repo.manifest.dirlog(dir)
@@ -213,6 +214,8 @@ 
             label = dir
             revlogfiles = mf.files()
             storefiles.difference_update(revlogfiles)
+            if progress: # should be true since we're in a subdirectory
+                progress()
         if self.refersmf:
             # Do not check manifest if there are only changelog entries with
             # null manifests.
@@ -263,19 +266,29 @@ 
         if not dir and subdirnodes:
             self.ui.status(_("checking directory manifests\n"))
             storefiles = set()
+            subdirs = set()
             revlogv1 = self.revlogv1
             for f, f2, size in repo.store.datafiles():
                 if not f:
                     self.err(None, _("cannot decode filename '%s'") % f2)
                 elif (size > 0 or not revlogv1) and f.startswith('meta/'):
                     storefiles.add(_normpath(f))
+                    subdirs.add(os.path.dirname(f))
+            subdircount = len(subdirs)
+            currentsubdir = [0]
+            def progress():
+                currentsubdir[0] += 1
+                ui.progress(_('checking'), currentsubdir[0], total=subdircount,
+                            unit=_('manifests'))

         for subdir, linkrevs in subdirnodes.iteritems():
-            subdirfilenodes = self._verifymanifest(linkrevs, subdir,
storefiles)
+            subdirfilenodes = self._verifymanifest(linkrevs, subdir,
storefiles,
+                                                   progress)
             for f, onefilenodes in subdirfilenodes.iteritems():
                 filenodes.setdefault(f, {}).update(onefilenodes)

         if not dir and subdirnodes:
+            ui.progress(_('checking'), None)
             for f in sorted(storefiles):