Patchwork [1,of,3] revlog: ignore empty trailing chunks when reading segments

login
register
mail settings
Submitter Paul Morelle
Date Oct. 16, 2017, 4:35 p.m.
Message ID <ac3901a97e195627a2ed.1508171748@taranis.localdomain>
Download mbox | patch
Permalink /patch/25006/
State Accepted
Headers show

Comments

Paul Morelle - Oct. 16, 2017, 4:35 p.m.
# HG changeset patch
# User Paul Morelle <paul.morelle@octobus.net>
# Date 1507554821 -7200
#      Mon Oct 09 15:13:41 2017 +0200
# Node ID ac3901a97e195627a2ed4e65040912326ce5d943
# Parent  a652b7763f669683eb5540c6d4b77ee18e55bc80
# EXP-Topic optimized-read
# Available At https://bitbucket.org/octobus/mercurial-devel/
#              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r ac3901a97e19
revlog: ignore empty trailing chunks when reading segments

When a merge commit creates an empty diff in the revlog, its offset may still
be quite far from the end of the previous chunk.
Skipping these empty chunks may reduce read size significantly.
In most cases, there is no gain, and in some cases, little gain.
On my clone of pypy, `hg manifest` reads 65% less bytes (96140 i/o 275943) for
revision 4260 by ignoring the only empty trailing diff.
For revision 2229, 35% (34557 i/o 53435)
Sadly, this is difficult to reproduce, as hg clone can make its own different
structure every time.

Patch

diff -r a652b7763f66 -r ac3901a97e19 mercurial/revlog.py
--- a/mercurial/revlog.py	Sat Oct 14 12:03:42 2017 -0400
+++ b/mercurial/revlog.py	Mon Oct 09 15:13:41 2017 +0200
@@ -1327,8 +1327,14 @@ 
         l = []
         ladd = l.append
 
+        firstrev = revs[0]
+        # Skip trailing revisions with empty diff
+        for lastrev in revs[::-1]:
+            if length(lastrev) != 0:
+                break
+
         try:
-            offset, data = self._getsegmentforrevs(revs[0], revs[-1], df=df)
+            offset, data = self._getsegmentforrevs(firstrev, lastrev, df=df)
         except OverflowError:
             # issue4215 - we can't cache a run of chunks greater than
             # 2G on Windows