Patchwork [10,of,10,V2] sparse-revlog: put the native implementation of slicechunktodensity to use

login
register
mail settings
Submitter Boris Feld
Date Nov. 15, 2018, 10:38 a.m.
Message ID <878a0fd4a339d39211f7.1542278328@localhost.localdomain>
Download mbox | patch
Permalink /patch/36599/
State Accepted
Headers show

Comments

Boris Feld - Nov. 15, 2018, 10:38 a.m.
# HG changeset patch
# User Boris Feld <boris.feld@octobus.net>
# Date 1542276698 -3600
#      Thu Nov 15 11:11:38 2018 +0100
# Node ID 878a0fd4a339d39211f7648ad7da9023bee40a2b
# Parent  2e4071c2d69873431c32d71e8e82661e641dc821
# EXP-Topic sparse-perf
# Available At https://bitbucket.org/octobus/mercurial-devel/
#              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 878a0fd4a339
sparse-revlog: put the native implementation of slicechunktodensity to use

When possible, the C implementation of delta chain slicing will be used.
providing a large boost in performance for this operation.

To take a practical example of restoring manifest revision '59547c40bc4c' for
a reference NetBeans repository (using sparse-revlog). The media time of the
step `slice-sparse-chain` of `perfrevlogrevision` improve from 0.660 ms to
0.098 ms;

The full series move delta chain slicing from 1.120 ms to 0.098 ms;

Implementing _slicechunktosize into C would yield further improvements.
However, the performance seems good enough for now.

Patch

diff --git a/mercurial/revlogutils/deltas.py b/mercurial/revlogutils/deltas.py
--- a/mercurial/revlogutils/deltas.py
+++ b/mercurial/revlogutils/deltas.py
@@ -44,6 +44,7 @@  class _testrevlog(object):
         self._srdensitythreshold = density
         self._srmingapsize = mingap
         self._snapshot = set(snapshot)
+        self.index = None
 
     def start(self, rev):
         if rev == 0:
@@ -120,9 +121,12 @@  def slicechunk(revlog, revs, targetsize=
         targetsize = max(targetsize, revlog._srmingapsize)
     # targetsize should not be specified when evaluating delta candidates:
     # * targetsize is used to ensure we stay within specification when reading,
-    for chunk in _slicechunktodensity(revlog, revs,
-                                      revlog._srdensitythreshold,
-                                      revlog._srmingapsize):
+    densityslicing = getattr(revlog.index, 'slicechunktodensity', None)
+    if densityslicing is None:
+        densityslicing = lambda x, y, z: _slicechunktodensity(revlog, x, y, z)
+    for chunk in densityslicing(revs,
+                                revlog._srdensitythreshold,
+                                revlog._srmingapsize):
         for subchunk in _slicechunktosize(revlog, chunk, targetsize):
             yield subchunk