Patchwork D4479: [RFC] obsolete: add mechanism to filter obsstore

login
register
mail settings
Submitter phabricator
Date Sept. 4, 2018, 11:19 p.m.
Message ID <differential-rev-PHID-DREV-zjmhzxdpimtyjj3to3mq-req@phab.mercurial-scm.org>
Download mbox | patch
Permalink /patch/34320/
State New
Headers show

Comments

phabricator - Sept. 4, 2018, 11:19 p.m.
lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This patch is not ready to land and only sent for discussion.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4479

AFFECTED FILES
  mercurial/obsolete.py

CHANGE DETAILS




To: lothiraldan, #hg-reviewers
Cc: mercurial-devel
phabricator - Sept. 4, 2018, 11:22 p.m.
martinvonz added a comment.


  Why? (Sorry if I'm asking to early and you were just about to send an update.)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4479

To: lothiraldan, #hg-reviewers
Cc: martinvonz, mercurial-devel
phabricator - Sept. 4, 2018, 11:29 p.m.
martinvonz added a comment.


  I'll reply to my own question: The context is in https://phab.mercurial-scm.org/D2679.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4479

To: lothiraldan, #hg-reviewers
Cc: martinvonz, mercurial-devel
phabricator - Sept. 5, 2018, 9:45 p.m.
pulkit added a comment.


  I just had a quick look. obsmarker repr hash, are we kind of inventing a way to index obsmarkers? I am pretty sure you people have thought about that, can you explain why you went with obsmarker repr hash instead of including a real index?
  
  Also, feel free to point me to old discussions if this was discussed earlier.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4479

To: lothiraldan, #hg-reviewers
Cc: pulkit, martinvonz, mercurial-devel

Patch

diff --git a/mercurial/obsolete.py b/mercurial/obsolete.py
--- a/mercurial/obsolete.py
+++ b/mercurial/obsolete.py
@@ -70,6 +70,7 @@ 
 from __future__ import absolute_import
 
 import errno
+import hashlib
 import struct
 
 from .i18n import _
@@ -461,7 +462,7 @@ 
     return _unpack('>B', data[0:1])[0]
 
 @util.nogc
-def _readmarkers(data, off=None, stop=None):
+def _readmarkers(data, off=None, stop=None, filterfunc=None):
     """Read and enumerate markers from raw data"""
     diskversion = _readmarkerversion(data)
     if not off:
@@ -471,7 +472,10 @@ 
     if diskversion not in formats:
         msg = _('parsing obsolete marker: unknown version %r') % diskversion
         raise error.UnknownVersion(msg, version=diskversion)
-    return diskversion, formats[diskversion][0](data, off, stop)
+    markers = formats[diskversion][0](data, off, stop)
+    if filterfunc:
+        markers = filterfunc(markers)
+    return diskversion, markers
 
 def encodeheader(version=_fm0version):
     return _pack('>B', version)
@@ -515,6 +519,19 @@ 
             raise error.Abort(_('bad obsolescence marker detected: '
                                'invalid successors nullid'))
 
+def _filtermarkers(data):
+    """
+    Return a generator that will filter markers based on the data param passed.
+    data: a newline-separated list of obsmarker repr hashes
+    """
+    filterset = set(filter(lambda x: x != '', data.split('\n')))
+    def filtermarkers(markers):
+        for marker in markers:
+            markerhash = hashlib.md5(marker.__repr__()).hexdigest()
+            if markerhash not in filterset:
+                yield marker
+    return filtermarkers
+
 class obsstore(object):
     """Store obsolete markers
 
@@ -676,7 +693,8 @@ 
         data = self._data
         if not data:
             return []
-        self._version, markers = _readmarkers(data)
+        filterfunc = _filtermarkers(self.svfs.tryread('obsfilter'))
+        self._version, markers = _readmarkers(data, filterfunc=filterfunc)
         markers = list(markers)
         _checkinvalidmarkers(markers)
         return markers