Patchwork [STABLE] largefiles: fix commit of a directory with no largefile changes (issue4330)

login
register
mail settings
Submitter Matt Harbison
Date Jan. 18, 2015, 9:28 p.m.
Message ID <c27d3218ffa9096bb081.1421616485@Envy>
Download mbox | patch
Permalink /patch/7515/
State Accepted
Commit ab6fd3205dadcd5985658febb3738425a9fac710
Headers show

Comments

Matt Harbison - Jan. 18, 2015, 9:28 p.m.
# HG changeset patch
# User Matt Harbison <matt_harbison@yahoo.com>
# Date 1421612140 18000
#      Sun Jan 18 15:15:40 2015 -0500
# Branch stable
# Node ID c27d3218ffa9096bb0812e30d5e66a05b88a92d6
# Parent  cb21ffd4b6c7e6572e6ab33fb3dfb18949b1ecd7
largefiles: fix commit of a directory with no largefile changes (issue4330)

When a directory is named in the commit file list, the previous behavior was to
walk the list, and if no normal files in the directory were also named, add the
corresponding standin for each largefile in that directory.  The directory is
then dropped from the list, so that committing a directory with no normal file
changes works.  It then added the corresponding standin directory for the first
largefile seen, by prefixing it with '.hglf/'.

The latter is unnecessary since each affected largefile is explicitly referenced
by its standin in the list.  It also caused an abort if there were no changed
largefiles in the directory, because none of its standins changed:

    abort: .hglf/foo/bar: no match under directory!

This list of files is used to tweak a matcher in lfutil.updatestandinsbymatch(),
which is what is passed to commit().

The status() call that is ultimately done in the commit code with this matcher
seems to have some OS specific differences.  It is not necessary to append '.'
for Windows to run the largefiles tests cleanly.  But if '.' is not added to the
list, the match function isn't called on Linux, so status() would miss any
normal files that were also in a named directory.  The commit then proceeds
without those normal files, or says "nothing changed" if there were no changed
largefiles in the directory.  This is not filesystem specific, as VFAT on Linux
had the same behavior as when run on ext4.  It is also not an issue with
lfilesrepo.status(), since that only calls the overridden implementation when
paths are passed to commit.  I dont have access to an OS X machine ATM to test
there.

Maybe there's a better way to do this.  But since the standin directory for the
first largefile was previously being added, and that caused the same walk in
status(), there's no preformance change to this.  There is no danger of
erroneously committing files in '.', because the original match function is
called, and if it fails, the lfutil.updatestandinsbymatch() tweaked matcher only
indicates a match if the file is in the list of standins- and '.' never is.  The
added tests confirm this.
Matt Mackall - Jan. 19, 2015, 12:29 a.m.
On Sun, 2015-01-18 at 16:28 -0500, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison <matt_harbison@yahoo.com>
> # Date 1421612140 18000
> #      Sun Jan 18 15:15:40 2015 -0500
> # Branch stable
> # Node ID c27d3218ffa9096bb0812e30d5e66a05b88a92d6
> # Parent  cb21ffd4b6c7e6572e6ab33fb3dfb18949b1ecd7
> largefiles: fix commit of a directory with no largefile changes (issue4330)

This is queued for stable, thanks.
Matt Harbison - Jan. 19, 2015, 12:49 a.m.
On Sun, 18 Jan 2015 19:29:51 -0500, Matt Mackall <mpm@selenic.com> wrote:

> On Sun, 2015-01-18 at 16:28 -0500, Matt Harbison wrote:
>> # HG changeset patch
>> # User Matt Harbison <matt_harbison@yahoo.com>
>> # Date 1421612140 18000
>> #      Sun Jan 18 15:15:40 2015 -0500
>> # Branch stable
>> # Node ID c27d3218ffa9096bb0812e30d5e66a05b88a92d6
>> # Parent  cb21ffd4b6c7e6572e6ab33fb3dfb18949b1ecd7
>> largefiles: fix commit of a directory with no largefile changes  
>> (issue4330)
>
> This is queued for stable, thanks.
>

Is the OS specific status behavior expected, w.r.t. whether match() is  
called when no directories are named?  Any status expert that needs to be  
poked to look at this?  I wasn't able to make anything fail from the  
command line, so maybe it's just an optimization.

--Matt
Matt Mackall - Jan. 20, 2015, 9:47 p.m.
On Sun, 2015-01-18 at 19:49 -0500, Matt Harbison wrote:
> On Sun, 18 Jan 2015 19:29:51 -0500, Matt Mackall <mpm@selenic.com> wrote:
> 
> > On Sun, 2015-01-18 at 16:28 -0500, Matt Harbison wrote:
> >> # HG changeset patch
> >> # User Matt Harbison <matt_harbison@yahoo.com>
> >> # Date 1421612140 18000
> >> #      Sun Jan 18 15:15:40 2015 -0500
> >> # Branch stable
> >> # Node ID c27d3218ffa9096bb0812e30d5e66a05b88a92d6
> >> # Parent  cb21ffd4b6c7e6572e6ab33fb3dfb18949b1ecd7
> >> largefiles: fix commit of a directory with no largefile changes  
> >> (issue4330)
> >
> > This is queued for stable, thanks.
> >
> 
> Is the OS specific status behavior expected, w.r.t. whether match() is  
> called when no directories are named?

Nope. At a guess, the Windows version of the behavior is probably the
unintended one.
Matt Harbison - Feb. 1, 2015, 6:54 p.m.
On Tue, 20 Jan 2015 16:47:14 -0500, Matt Mackall <mpm@selenic.com> wrote:

> On Sun, 2015-01-18 at 19:49 -0500, Matt Harbison wrote:
>> On Sun, 18 Jan 2015 19:29:51 -0500, Matt Mackall <mpm@selenic.com>  
>> wrote:
>>
>> > On Sun, 2015-01-18 at 16:28 -0500, Matt Harbison wrote:
>> >> # HG changeset patch
>> >> # User Matt Harbison <matt_harbison@yahoo.com>
>> >> # Date 1421612140 18000
>> >> #      Sun Jan 18 15:15:40 2015 -0500
>> >> # Branch stable
>> >> # Node ID c27d3218ffa9096bb0812e30d5e66a05b88a92d6
>> >> # Parent  cb21ffd4b6c7e6572e6ab33fb3dfb18949b1ecd7
>> >> largefiles: fix commit of a directory with no largefile changes
>> >> (issue4330)
>> >
>> > This is queued for stable, thanks.
>> >
>>
>> Is the OS specific status behavior expected, w.r.t. whether match() is
>> called when no directories are named?
>
> Nope. At a guess, the Windows version of the behavior is probably the
> unintended one.
>

For the benefit of anybody searching the archives in the future, this  
isn't OS dependent- it is filesystem dependent.  I messed up the Linux  
vfat setup, and didn't actually run it on a vfat fs.  When I did, it  
exhibited the same behavior as Windows.

What happens is that in dirstate.walk(), running on a case insensitive  
filesystem forces step 3 to run, which passes every single file in  
dirstate to the matcher's match function.  Any matches are returned from  
walk().  If there are no directories in the matcher's file list, step 3 is  
skipped on case sensitive filesystems.  Thus the need to add '.' to force  
the full walk.

The code as committed is correct, but I'll update the comment after  
default is reopened.

--Matt

Patch

diff --git a/hgext/largefiles/reposetup.py b/hgext/largefiles/reposetup.py
--- a/hgext/largefiles/reposetup.py
+++ b/hgext/largefiles/reposetup.py
@@ -326,7 +326,12 @@ 
                         if self.dirstate.normalize(lf).startswith(d):
                             actualfiles.append(lf)
                             if not matcheddir:
-                                actualfiles.append(lfutil.standin(f))
+                                # There may still be normal files in the dir, so
+                                # make sure a directory is in the list, which
+                                # forces status to walk and call the match
+                                # function on the matcher.  Windows does NOT
+                                # require this.
+                                actualfiles.append('.')
                                 matcheddir = True
                 # Nothing in dir, so readd it
                 # and let commit reject it
diff --git a/tests/test-subrepo-deep-nested-change.t b/tests/test-subrepo-deep-nested-change.t
--- a/tests/test-subrepo-deep-nested-change.t
+++ b/tests/test-subrepo-deep-nested-change.t
@@ -351,5 +351,30 @@ 
   R sub1/sub2/test.txt
   ? foo/bar/abc
   ? sub1/sub2/untracked.txt
+  $ hg add sub1/sub2
+  $ hg ci -Sqm 'forget testing'
+
+Test issue4330: commit a directory where only normal files have changed
+  $ touch foo/bar/large.dat
+  $ hg add --large foo/bar/large.dat
+  $ hg ci -m 'add foo/bar/large.dat'
+  $ touch a.txt
+  $ touch a.dat
+  $ hg add -v foo/bar/abc a.txt a.dat
+  adding a.dat as a largefile
+  adding a.txt
+  adding foo/bar/abc (glob)
+  $ hg ci -m 'dir commit with only normal file deltas' foo/bar
+  $ hg status
+  A a.dat
+  A a.txt
+
+Test a directory commit with a changed largefile and a changed normal file
+  $ echo changed > foo/bar/large.dat
+  $ echo changed > foo/bar/abc
+  $ hg ci -m 'dir commit with normal and lf file deltas' foo
+  $ hg status
+  A a.dat
+  A a.txt
 
   $ cd ..