Patchwork [V2] manifest: make lru size configurable

login
register
mail settings
Submitter Durham Goode
Date Feb. 4, 2015, 2:44 a.m.
Message ID <7e634219a049f21540d5.1423017849@dev2000.prn2.facebook.com>
Download mbox | patch
Permalink /patch/7645/
State Accepted
Commit ed5e8a9598cea22b20503ef969cc796ad9a71ea9
Headers show

Comments

Durham Goode - Feb. 4, 2015, 2:44 a.m.
# HG changeset patch
# User Durham Goode <durham@fb.com>
# Date 1422061563 28800
#      Fri Jan 23 17:06:03 2015 -0800
# Node ID 7e634219a049f21540d5803d8c0d8a8d25d809dd
# Parent  e1dbe0b215ae137eec53ceb12440536d570a83d2
manifest: make lru size configurable

On machines with lots of ram, it's beneficial to increase the lru size of the
manifest cache.  On a large repo, configuring the lru to be size 10 can shave a
large rebase (~12 commits) down from 95s to 70s.
Matt Mackall - Feb. 4, 2015, 9:40 p.m.
On Tue, 2015-02-03 at 18:44 -0800, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1422061563 28800
> #      Fri Jan 23 17:06:03 2015 -0800
> # Node ID 7e634219a049f21540d5803d8c0d8a8d25d809dd
> # Parent  e1dbe0b215ae137eec53ceb12440536d570a83d2
> manifest: make lru size configurable

Queued for default, thanks.

> On machines with lots of ram, it's beneficial to increase the lru size of the
> manifest cache.  On a large repo, configuring the lru to be size 10 can shave a
> large rebase (~12 commits) down from 95s to 70s.

Any chance I could get you to generate a list of manifests visited on
that rebase? Really curious why the working set is larger than 4. Seems
like we're probably doing something silly.

Patch

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -323,6 +323,9 @@  class localrepository(object):
         maxchainlen = self.ui.configint('format', 'maxchainlen')
         if maxchainlen is not None:
             self.svfs.options['maxchainlen'] = maxchainlen
+        manifestcachesize = self.ui.configint('format', 'manifestcachesize')
+        if manifestcachesize is not None:
+            self.svfs.options['manifestcachesize'] = manifestcachesize
 
     def _writerequirements(self):
         reqfile = self.vfs("requires", "w")
diff --git a/mercurial/manifest.py b/mercurial/manifest.py
--- a/mercurial/manifest.py
+++ b/mercurial/manifest.py
@@ -220,9 +220,14 @@  def _parse(lines):
 
 class manifest(revlog.revlog):
     def __init__(self, opener):
-        # we expect to deal with not more than four revs at a time,
-        # during a commit --amend
-        self._mancache = util.lrucachedict(4)
+        # During normal operations, we expect to deal with not more than four
+        # revs at a time (such as during commit --amend). When rebasing large
+        # stacks of commits, the number can go up, hence the config knob below.
+        cachesize = 4
+        opts = getattr(opener, 'options', None)
+        if opts is not None:
+            cachesize = opts.get('manifestcachesize', cachesize)
+        self._mancache = util.lrucachedict(cachesize)
         revlog.revlog.__init__(self, opener, "00manifest.i")
 
     def readdelta(self, node):