Patchwork [1,of,6] lfs: add a blob verification method to the local store

login
register
mail settings
Submitter Matt Harbison
Date March 19, 2018, 4:08 a.m.
Message ID <793636c45477f81af6fd.1521432504@Envy>
Download mbox | patch
Permalink /patch/29600/
State Accepted
Headers show

Comments

Matt Harbison - March 19, 2018, 4:08 a.m.
# HG changeset patch
# User Matt Harbison <matt_harbison@yahoo.com>
# Date 1519360432 18000
#      Thu Feb 22 23:33:52 2018 -0500
# Node ID 793636c45477f81af6fddacc458979fecc84ae3a
# Parent  2fc97972bc88809f86bbd91fbfe8582ab59c3ef7
lfs: add a blob verification method to the local store

A corrupt blob can be signaled through the Batch API response, without actually
transferring the file.  A true/false indicator is slightly easier than
immediately catching an exception.
Matt Harbison - March 27, 2018, 12:45 a.m.
On Mon, 19 Mar 2018 00:08:24 -0400, Matt Harbison <mharbison72@gmail.com>  
wrote:

> # HG changeset patch
> # User Matt Harbison <matt_harbison@yahoo.com>
> # Date 1519360432 18000
> #      Thu Feb 22 23:33:52 2018 -0500
> # Node ID 793636c45477f81af6fddacc458979fecc84ae3a
> # Parent  2fc97972bc88809f86bbd91fbfe8582ab59c3ef7
> lfs: add a blob verification method to the local store

gentle ping on this series
Yuya Nishihara - March 29, 2018, 11:57 a.m.
On Mon, 19 Mar 2018 00:08:24 -0400, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison <matt_harbison@yahoo.com>
> # Date 1519360432 18000
> #      Thu Feb 22 23:33:52 2018 -0500
> # Node ID 793636c45477f81af6fddacc458979fecc84ae3a
> # Parent  2fc97972bc88809f86bbd91fbfe8582ab59c3ef7
> lfs: add a blob verification method to the local store
> 
> A corrupt blob can be signaled through the Batch API response, without actually
> transferring the file.  A true/false indicator is slightly easier than
> immediately catching an exception.
> 
> diff --git a/hgext/lfs/blobstore.py b/hgext/lfs/blobstore.py
> --- a/hgext/lfs/blobstore.py
> +++ b/hgext/lfs/blobstore.py
> @@ -175,6 +175,17 @@ class local(object):
>              _verify(oid, blob)
>          return blob
>  
> +    def verify(self, oid):
> +        """Indicate whether or not the hash of the underlying file matches its
> +        name."""
> +        sha256 = hashlib.sha256()
> +
> +        with self.open(oid) as fp:
> +            for chunk in util.filechunkiter(fp, size=1048576):
> +                sha256.update(chunk)
> +
> +        return oid == sha256.hexdigest()

Queued, but can you eliminate copypasta? This is quite similar to
_verifyfile().
Yuya Nishihara - March 29, 2018, 12:43 p.m.
On Mon, 26 Mar 2018 20:45:07 -0400, Matt Harbison wrote:
> On Mon, 19 Mar 2018 00:08:24 -0400, Matt Harbison <mharbison72@gmail.com>  
> wrote:
> 
> > # HG changeset patch
> > # User Matt Harbison <matt_harbison@yahoo.com>
> > # Date 1519360432 18000
> > #      Thu Feb 22 23:33:52 2018 -0500
> > # Node ID 793636c45477f81af6fddacc458979fecc84ae3a
> > # Parent  2fc97972bc88809f86bbd91fbfe8582ab59c3ef7
> > lfs: add a blob verification method to the local store
> 
> gentle ping on this series

I'm not comfortable to queue these with no test coverage, but the lfs is
still experimental and I didn't find any critical issues. So queued 1-5,
thanks.

I expect indygreg will double-check these patches.

Patch

diff --git a/hgext/lfs/blobstore.py b/hgext/lfs/blobstore.py
--- a/hgext/lfs/blobstore.py
+++ b/hgext/lfs/blobstore.py
@@ -175,6 +175,17 @@  class local(object):
             _verify(oid, blob)
         return blob
 
+    def verify(self, oid):
+        """Indicate whether or not the hash of the underlying file matches its
+        name."""
+        sha256 = hashlib.sha256()
+
+        with self.open(oid) as fp:
+            for chunk in util.filechunkiter(fp, size=1048576):
+                sha256.update(chunk)
+
+        return oid == sha256.hexdigest()
+
     def has(self, oid):
         """Returns True if the local blobstore contains the requested blob,
         False otherwise."""