Patchwork [22,of,22] transaction: flush obsstore indexes on success

login
register
mail settings
Submitter Jun Wu
Date June 4, 2017, 11:59 p.m.
Message ID <c800204c54a7fa939d5b.1496620774@x1c>
Download mbox | patch
Permalink /patch/21201/
State Accepted
Headers show

Comments

Jun Wu - June 4, 2017, 11:59 p.m.
# HG changeset patch
# User Jun Wu <quark@fb.com>
# Date 1496620376 25200
#      Sun Jun 04 16:52:56 2017 -0700
# Node ID c800204c54a7fa939d5bdacacd94ec1146e18b90
# Parent  cd962d45b21f4a2c20bf60726b4b3e6c6e136aca
# Available At https://bitbucket.org/quark-zju/hg-draft
#              hg pull https://bitbucket.org/quark-zju/hg-draft -r c800204c54a7
transaction: flush obsstore indexes on success

This speeds up all four revsets (obsolete, bumped, divergent, unstable)
significantly since we won't parse all markers from obsstore.

With the indexes built (repo.obsstore.flushindexes()), on hg-committed,
hg perfvolatilesets --clear-obsstore shows all revsets can be calculated
within 0.08 seconds. Cheers!

             | old [1]  | before   | after
  --------------------------------------------
   bumped    | 0.791746 | 0.464715 | 0.037455
   divergent | 0.823026 | 0.524332 | 0.071759
   extinct   | 0.597182 | 0.409694 | 0.047234
   obsolete  | 0.583754 | 0.378805 | 0.017877
   suspended | 0.613364 | 0.434053 | 0.047443
   unstable  | 0.579994 | 0.394650 | 0.027899
   base      | 0.001387 | 0.001205 | 0.001042
   immutable | 0.006587 | 0.006494 | 0.005739
   served    | 0.469272 | 0.369578 | 0.024549
   visible   | 0.573799 | 0.392960 | 0.021543

Note: "hg perfvolatilesets" without "--clear-obsolete" is not expected to
change by this patch. And it is less meaningful because its pattern won't
happen in real-world use-cases.

               | old [1] | before | after
  -----------------------------------------
   hg id       | 0.73    | 0.55   | 0.199
   hg log -r . | 1.05    | 0.74   | 0.274

A downside is this will slow down commit time. My test shows it's like 0.1s
on hg-committed, which seems fine. It'll be a no-op if obsstore does not
change.

[1]: before "obsstore: use radixlink to back markerindex"

Patch

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1269,4 +1269,7 @@  class localrepository(object):
             branchmap.updatecache(self.filtered('served'))
 
+        if self.obsstore:
+            self.obsstore.flushindexes()
+
     def invalidatecaches(self):
 
diff --git a/mercurial/obsolete.py b/mercurial/obsolete.py
--- a/mercurial/obsolete.py
+++ b/mercurial/obsolete.py
@@ -840,4 +840,9 @@  class obsstore(object):
             self.svfs.tryunlink('cache/obsindex-%s' % name)
 
+    def flushindexes(self):
+        """call this on a successful transaction. dump indexes to disk"""
+        for name in ('successors', 'precursors', 'children'):
+            getattr(self, name).flush()
+
     def relevantmarkers(self, nodes):
         """return a set of all obsolescence markers relevant to a set of nodes.