Patchwork D8723: fix: prefetch file contents

login
register
mail settings
Submitter phabricator
Date July 10, 2020, 6:45 a.m.
Message ID <differential-rev-PHID-DREV-6w4wpyezmkp2okzclzvm-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/46686/
State Superseded
Headers show

Comments

phabricator - July 10, 2020, 6:45 a.m.
rdamazio created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  This prevents the worker subprocesses from contacting the server individually,
  which is either inefficient, or leads to problems if the connection is shared
  among them.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D8723

AFFECTED FILES
  hgext/fix.py

CHANGE DETAILS




To: rdamazio, #hg-reviewers
Cc: mercurial-patches, mercurial-devel

Patch

diff --git a/hgext/fix.py b/hgext/fix.py
--- a/hgext/fix.py
+++ b/hgext/fix.py
@@ -271,6 +271,11 @@ 
         basepaths = getbasepaths(repo, opts, workqueue, basectxs)
         fixers = getfixers(ui)
 
+        # Rather than letting each worker independently fetch the files
+        # (which also would add complications for shared/keepalive
+        # connections), prefetch them all first.
+        _prefetchfiles(repo, workqueue, basepaths)
+
         # There are no data dependencies between the workers fixing each file
         # revision, so we can use all available parallelism.
         def getfixes(items):
@@ -630,6 +635,24 @@ 
                 basectxs[rev].add(pctx)
     return basectxs
 
+def _prefetchfiles(repo, workqueue, basepaths):
+    toprefetch = set()
+
+    # Prefetch the files that will be fixed.
+    for rev, path in workqueue:
+        if rev == wdirrev:
+            continue
+        toprefetch.add((rev, path))
+
+    # Prefetch the base contents for lineranges().
+    for (baserev, fixrev, path), basepath in basepaths.items():
+        toprefetch.add((baserev, basepath))
+
+    if toprefetch:
+        scmutil.prefetchfiles(repo, [
+            (rev, scmutil.matchfiles(repo, [path])) for rev, path in toprefetch
+        ])
+
 
 def fixfile(ui, repo, opts, fixers, fixctx, path, basepaths, basectxs):
     """Run any configured fixers that should affect the file in this context