Patchwork [1,of,4,V2] lfs: add the ability to disable the usercache

login
register
mail settings
Submitter Matt Harbison
Date April 10, 2018, 8:09 p.m.
Message ID <473a8954c957945d1642.1523390963@MATT7H-PC.attotech.com>
Download mbox | patch
Permalink /patch/30644/
State Accepted
Headers show

Comments

Matt Harbison - April 10, 2018, 8:09 p.m.
# HG changeset patch
# User Matt Harbison <matt_harbison@yahoo.com>
# Date 1523155211 14400
#      Sat Apr 07 22:40:11 2018 -0400
# Node ID 473a8954c957945d1642d08eeefc8fe706597b59
# Parent  49a8c2cc7978cdd73155fc354aae8851a79c58d8
lfs: add the ability to disable the usercache

While the usercache is important for real world uses, I've been tripped up more
than a couple of times by it in tests- thinking a file was being downloaded, but
it was simply linked from the local cache.  The syntax for setting it is the
same as for setting a null remote endpoint, and like that endpoint, is left
undocumented.

This may or may not be a useful feature in the real world (I'd expect any sane
filesystem to support hardlinks at this point).
Yuya Nishihara - April 11, 2018, 11:34 a.m.
On Tue, 10 Apr 2018 16:09:23 -0400, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison <matt_harbison@yahoo.com>
> # Date 1523155211 14400
> #      Sat Apr 07 22:40:11 2018 -0400
> # Node ID 473a8954c957945d1642d08eeefc8fe706597b59
> # Parent  49a8c2cc7978cdd73155fc354aae8851a79c58d8
> lfs: add the ability to disable the usercache

Queued, thanks.

Patch

diff --git a/hgext/lfs/blobstore.py b/hgext/lfs/blobstore.py
--- a/hgext/lfs/blobstore.py
+++ b/hgext/lfs/blobstore.py
@@ -7,6 +7,7 @@ 
 
 from __future__ import absolute_import
 
+import errno
 import hashlib
 import json
 import os
@@ -59,6 +60,26 @@  class lfsvfs(vfsmod.vfs):
 
         yield ('', [], oids)
 
+class nullvfs(lfsvfs):
+    def __init__(self):
+        pass
+
+    def exists(self, oid):
+        return False
+
+    def read(self, oid):
+        # store.read() calls into here if the blob doesn't exist in its
+        # self.vfs.  Raise the same error as a normal vfs when asked to read a
+        # file that doesn't exist.  The only difference is the full file path
+        # isn't available in the error.
+        raise IOError(errno.ENOENT, '%s: No such file or directory' % oid)
+
+    def walk(self, path=None, onerror=None):
+        return ('', [], [])
+
+    def write(self, oid, data):
+        pass
+
 class filewithprogress(object):
     """a file-like object that supports __len__ and read.
 
@@ -97,8 +118,14 @@  class local(object):
     def __init__(self, repo):
         fullpath = repo.svfs.join('lfs/objects')
         self.vfs = lfsvfs(fullpath)
-        usercache = lfutil._usercachedir(repo.ui, 'lfs')
-        self.cachevfs = lfsvfs(usercache)
+        usercache = util.url(lfutil._usercachedir(repo.ui, 'lfs'))
+        if usercache.scheme in (None, 'file'):
+            self.cachevfs = lfsvfs(usercache.localpath())
+        elif usercache.scheme == 'null':
+            self.cachevfs = nullvfs()
+        else:
+            raise error.Abort(_('unknown lfs cache scheme: %s')
+                              % usercache.scheme)
         self.ui = repo.ui
 
     def open(self, oid):
@@ -129,11 +156,7 @@  class local(object):
             if realoid != oid:
                 raise error.Abort(_('corrupt remote lfs object: %s') % oid)
 
-        # XXX: should we verify the content of the cache, and hardlink back to
-        # the local store on success, but truncate, write and link on failure?
-        if not self.cachevfs.exists(oid):
-            self.ui.note(_('lfs: adding %s to the usercache\n') % oid)
-            lfutil.link(self.vfs.join(oid), self.cachevfs.join(oid))
+        self._linktousercache(oid)
 
     def write(self, oid, data):
         """Write blob to local blobstore.
@@ -144,9 +167,13 @@  class local(object):
         with self.vfs(oid, 'wb', atomictemp=True) as fp:
             fp.write(data)
 
+        self._linktousercache(oid)
+
+    def _linktousercache(self, oid):
         # XXX: should we verify the content of the cache, and hardlink back to
         # the local store on success, but truncate, write and link on failure?
-        if not self.cachevfs.exists(oid):
+        if (not self.cachevfs.exists(oid)
+            and not isinstance(self.cachevfs, nullvfs)):
             self.ui.note(_('lfs: adding %s to the usercache\n') % oid)
             lfutil.link(self.vfs.join(oid), self.cachevfs.join(oid))
 
diff --git a/tests/test-lfs-serve.t b/tests/test-lfs-serve.t
--- a/tests/test-lfs-serve.t
+++ b/tests/test-lfs-serve.t
@@ -35,6 +35,7 @@  masked by the Internal Server Error mess
   $ cat >> $HGRCPATH <<EOF
   > [lfs]
   > url=file:$TESTTMP/dummy-remote/
+  > usercache = null://
   > threshold=10
   > [web]
   > allow_push=*