From patchwork Sat Aug  3 01:27:10 2019
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: D6711: branchheads: store wdir-dependent caches in wcache (issue6181)
From: phabricator <phabricator@mercurial-scm.org>
X-Patchwork-Id: 41134
Message-Id: <differential-rev-PHID-DREV-globmffs4dlfltoytlck-req@mercurial-scm.org>
To: Phabricator <phabricator@mercurial-scm.org>
Cc: mercurial-devel@mercurial-scm.org
Date: Sat, 3 Aug 2019 01:27:10 +0000

spectral created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Previously, all branch2-* caches were stored in .hg/cache, which is shared
  across repos when using `hg share`. This can cause cache thrashing when the
  repos are accessed concurrently, since some of the caches depend on wdir state
  (which is NOT shared across the repos).
  
  There's already a cache directory for caches that depend on wdir state, wcache,
  so let's just put the caches there.
  
  This change does not clean up any existing caches in .hg/cache that are now
  moved to the new location, and it does not re-use the caches from that location
  when constructing them in the wcache directory. Upgrades to versions of
  Mercurial that have this commit will need to regenerate their branchheads once, and
  this may be an expensive operation.
  
  The format of these cache files is not changing - forwards/backwards
  compatibility remains; if a newer repo (with these in wcache) is accessed by an
  older hg, the older hg will either see the files that remain in .hg/cache, and
  update them there, or will not see the files at all and will fall back to a
  slower path (and then likely cache them in .hg/cache). If it's then accessed by
  a newer hg, the work done by the older hg is ignored, and the wcache copies are
  brought up to date.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6711

AFFECTED FILES
  mercurial/branchmap.py
  mercurial/utils/repoviewutil.py
  tests/test-branchmap-cache.t

CHANGE DETAILS




To: spectral, #hg-reviewers
Cc: mercurial-devel

diff --git a/tests/test-branchmap-cache.t b/tests/test-branchmap-cache.t
new file mode 100644
--- /dev/null
+++ b/tests/test-branchmap-cache.t
@@ -0,0 +1,38 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > share=
+  > [experimental]
+  > evolution=createmarkers
+  > EOF
+  $ hg init -q orig
+  $ cd orig
+  $ echo hi > foo
+  $ hg ci -qAm initial_rev0
+  $ echo "hi again" >> foo
+  $ hg ci -qAm "hi again rev1"
+  $ hg commit -q --amend -m "obsoleted 1, this is rev2"
+  $ cd ..
+  $ hg share -q orig other
+  $ cd other
+  $ hg co -qr 1 --hidden
+  updated to hidden changeset a3571a6d8234
+  (hidden revision 'a3571a6d8234' was rewritten as: 0444ce380f11)
+  $ cd ..
+
+Forcefully update the caches in each repo once; we're not using --debug since we
+know that these are going to update some/all of the caches, we don't need to
+know which.
+  $ hg -R orig debugupdatecache
+  $ hg -R other debugupdatecache
+
+The branchheads caches should not change just because we're accessing from a
+different repo each time. If they were stale, we'd get another line in the
+output per filtername that's stale.
+  $ hg -R orig debugupdatecache --debug
+  updating the branch cache
+  $ hg -R other debugupdatecache --debug
+  updating the branch cache
+  $ hg -R orig debugupdatecache --debug
+  updating the branch cache
+  $ hg -R other debugupdatecache --debug
+  updating the branch cache
diff --git a/mercurial/utils/repoviewutil.py b/mercurial/utils/repoviewutil.py
--- a/mercurial/utils/repoviewutil.py
+++ b/mercurial/utils/repoviewutil.py
@@ -20,3 +20,8 @@
                'served.hidden': 'served',
                'served': 'immutable',
                'immutable': 'base'}
+
+# List of views that are potentially dependent upon wdir state (such as the
+# current parents of the working directory) and need to be kept in wcache
+# instead of cache.
+wdirdependent = frozenset(['served', 'visible', 'visible-hidden'])
diff --git a/mercurial/branchmap.py b/mercurial/branchmap.py
--- a/mercurial/branchmap.py
+++ b/mercurial/branchmap.py
@@ -221,7 +221,7 @@
     def fromfile(cls, repo):
         f = None
         try:
-            f = repo.cachevfs(cls._filename(repo))
+            f = cls._opencachefile(repo)
             lineiter = iter(f)
             cachekey = next(lineiter).rstrip('\n').split(" ", 2)
             last, lrev = cachekey[:2]
@@ -271,12 +271,15 @@
                 self._closednodes.add(node)
 
     @staticmethod
-    def _filename(repo):
+    def _opencachefile(repo, *args, **kwargs):
         """name of a branchcache file for a given repo or repoview"""
         filename = "branch2"
+        vfs = repo.cachevfs
         if repo.filtername:
             filename = '%s-%s' % (filename, repo.filtername)
-        return filename
+            if repo.filtername in repoviewutil.wdirdependent:
+                vfs = repo.wcachevfs
+        return vfs(filename, *args, **kwargs)
 
     def validfor(self, repo):
         """Is the cache content valid regarding a repo
@@ -335,7 +338,7 @@
 
     def write(self, repo):
         try:
-            f = repo.cachevfs(self._filename(repo), "w", atomictemp=True)
+            f = self._opencachefile(repo, "w", atomictemp=True)
             cachekey = [hex(self.tipnode), '%d' % self.tiprev]
             if self.filteredhash is not None:
                 cachekey.append(hex(self.filteredhash))