Patchwork D7839: nodemap: deleted older raw data file when creating a new ones

login
register
mail settings
Submitter phabricator
Date Jan. 11, 2020, 5:04 p.m.
Message ID <differential-rev-PHID-DREV-ao64pck3rscrmfahkhua-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/44262/
State Superseded
Headers show

Comments

phabricator - Jan. 11, 2020, 5:04 p.m.
marmoute created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  When we write new full files, it replace an older one with a different name. We
  add the associated cleanup for the older file to be removed after the
  transaction.
  
  We delete all file matching the expected pattern to give use extra chance to
  delete orphan files we might have failed to delete earlier.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7839

AFFECTED FILES
  mercurial/revlogutils/nodemap.py
  tests/test-persistent-nodemap.t

CHANGE DETAILS




To: marmoute, #hg-reviewers
Cc: mercurial-devel
phabricator - Jan. 23, 2020, 9:32 p.m.
martinvonz added inline comments.

INLINE COMMENTS

> nodemap.py:151
> +    prefix = revlog.nodemap_file[:-2]
> +    pattern = re.compile(b"(^|/)%s-[0-9a-f]+.nd$" % prefix)
> +    new_file_path = _rawdata_filepath(revlog, uid)

meant to escape the "." in the regex?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7839/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7839

To: marmoute, #hg-reviewers
Cc: martinvonz, mercurial-devel

Patch

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -12,6 +12,8 @@ 
   $ hg debugbuilddag .+5000
   $ f --size .hg/store/00changelog.n
   .hg/store/00changelog.n: size=18
+  $ f --sha256 .hg/store/00changelog-*.nd
+  .hg/store/00changelog-????????????????.nd: sha256=5dbe62ab98a26668b544063d4d674ac4452ba903ee8895c52fd21d9bbd771e09 (glob)
   $ hg debugnodemap --dump-new | f --sha256 --size
   size=245760, sha256=5dbe62ab98a26668b544063d4d674ac4452ba903ee8895c52fd21d9bbd771e09
   $ hg debugnodemap --dump-disk | f --sha256 --bytes=256 --hexdump --size
@@ -32,3 +34,15 @@ 
   00d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   00e0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   00f0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
+
+add a new commit
+
+  $ hg up
+  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ echo foo > foo
+  $ hg add foo
+  $ hg ci -m 'foo'
+  $ f --size .hg/store/00changelog.n
+  .hg/store/00changelog.n: size=18
+  $ f --sha256 .hg/store/00changelog-*.nd --size
+  .hg/store/00changelog-????????????????.nd: size=245760, sha256=32d66a30ec934beecfbee8d3151d1f87b6d89ecc5713cdb123b4cdcd4e3b27a4 (glob)
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -9,6 +9,7 @@ 
 from __future__ import absolute_import
 
 import os
+import re
 import struct
 
 from .. import (
@@ -66,6 +67,16 @@ 
     data = persistent_data(revlog.index)
     uid = _make_uid()
     datafile = _rawdata_filepath(revlog, uid)
+    olds = _otherrawdata_filepath(revlog, uid)
+    if olds:
+        realvfs = getattr(revlog, '_realopener', revlog.opener)
+
+        def cleanup(tr):
+            for oldfile in olds:
+                realvfs.tryunlink(oldfile)
+
+        callback_id = b"revlog-cleanup-nodemap-%s" % revlog.nodemap_file
+        tr.addpostclose(callback_id, cleanup)
     # EXP-TODO: if this is a cache, this should use a cache vfs, not a
     # store vfs
     with revlog.opener(datafile, 'w') as fd:
@@ -129,6 +140,18 @@ 
     return b"%s-%s.nd" % (prefix, uid)
 
 
+def _otherrawdata_filepath(revlog, uid):
+    prefix = revlog.nodemap_file[:-2]
+    pattern = re.compile(b"(^|/)%s-[0-9a-f]+.nd$" % prefix)
+    newfilepath = _rawdata_filepath(revlog, uid)
+    dirpath = revlog.opener.dirname(newfilepath)
+    others = []
+    for f in revlog.opener.listdir(dirpath):
+        if pattern.match(f):
+            others.append(f)
+    return others
+
+
 ### Nodemap Trie
 #
 # This is a simple reference implementation to compute and serialise a nodemap