Patchwork subrepo: add status support for ignored and clean files in git subrepos

login
register
mail settings
Submitter Mathias De Maré
Date March 10, 2015, 6:08 p.m.
Message ID <16c867316afce2f57716.1426010917@mathias-Latitude-E6540>
Download mbox | patch
Permalink /patch/7971/
State Accepted
Commit e964edc3274eb16b6d8c264810020ecbd2402c90
Headers show

Comments

Mathias De Maré - March 10, 2015, 6:08 p.m.
# HG changeset patch
# User Mathias De Maré <mathias.demare@gmail.com>
# Date 1425935649 -3600
#      Mon Mar 09 22:14:09 2015 +0100
# Node ID 16c867316afce2f57716dbdf93dd5b0760186010
# Parent  02d7b5cd373bbb4e8263dad9bfbf9c4c3b0e4e3a
subrepo: add status support for ignored and clean files in git subrepos
Pierre-Yves David - March 10, 2015, 8:15 p.m.
On 03/10/2015 11:08 AM, mathias.demare@gmail.com wrote:
> # HG changeset patch
> # User Mathias De Maré <mathias.demare@gmail.com>
> # Date 1425935649 -3600
> #      Mon Mar 09 22:14:09 2015 +0100
> # Node ID 16c867316afce2f57716dbdf93dd5b0760186010
> # Parent  02d7b5cd373bbb4e8263dad9bfbf9c4c3b0e4e3a
> subrepo: add status support for ignored and clean files in git subrepos

Nothing seems too crazy here, but I would like a second opinion from 
someone that knowns something about either subrepo or git before goinf 
forward.
Matt Mackall - March 10, 2015, 10:20 p.m.
On Tue, 2015-03-10 at 19:08 +0100, mathias.demare@gmail.com wrote:
> # HG changeset patch
> # User Mathias De Maré <mathias.demare@gmail.com>
> # Date 1425935649 -3600
> #      Mon Mar 09 22:14:09 2015 +0100
> # Node ID 16c867316afce2f57716dbdf93dd5b0760186010
> # Parent  02d7b5cd373bbb4e8263dad9bfbf9c4c3b0e4e3a
> subrepo: add status support for ignored and clean files in git subrepos

Queued for default, thanks.
Matt Harbison - March 10, 2015, 11:35 p.m.
On Tue, 10 Mar 2015 14:08:37 -0400, <mathias.demare@gmail.com> wrote:

> # HG changeset patch
> # User Mathias De Maré <mathias.demare@gmail.com>
> # Date 1425935649 -3600
> #      Mon Mar 09 22:14:09 2015 +0100
> # Node ID 16c867316afce2f57716dbdf93dd5b0760186010
> # Parent  02d7b5cd373bbb4e8263dad9bfbf9c4c3b0e4e3a
> subrepo: add status support for ignored and clean files in git subrepos
>
> diff -r 02d7b5cd373b -r 16c867316afc mercurial/subrepo.py
> --- a/mercurial/subrepo.py	Tue Feb 10 15:59:12 2015 -0500
> +++ b/mercurial/subrepo.py	Mon Mar 09 22:14:09 2015 +0100
> @@ -1662,13 +1662,42 @@
>         deleted, unknown, ignored, clean = [], [], [], []
> +        command = ['status', '--porcelain', '-z']
>          if opts.get('unknown'):
> -            command = ['ls-files', '--others', '--exclude-standard']
> -            out = self._gitcommand(command)
> -            for line in out.split('\n'):
> -                if len(line) == 0:
> -                    continue
> -                unknown.append(line)
> +            command += ['--untracked-files=all']
> +        if opts.get('ignored'):
> +            command += ['--ignored']
> +        out = self._gitcommand(command)

Small nit: should this command execution be skipped if neither 'ignored'  
or 'unknown' are set?  (And possibly 'clean', but see below.)

> +
> +        changedfiles = set()
> +        changedfiles.update(modified)
> +        changedfiles.update(added)
> +        changedfiles.update(removed)
> +        for line in out.split('\0'):
> +            if not line:
> +                continue
> +            st = line[0:2]
> +            #moves and copies show 2 files on one line
> +            if line.find('\0') >= 0:
> +                filename1, filename2 = line[3:].split('\0')
> +            else:
> +                filename1 = line[3:]
> +                filename2 = None
> +
> +            changedfiles.add(filename1)
> +            if filename2:
> +                changedfiles.add(filename2)

I don't fully understand this, but it may be my lack of git knowledge.   
 From what I can understand from the git help on status, the file names are  
the source and destination.  But adding both would seem to block both from  
the clean list below.

Consider a clean working dir, and then 'hg cp foo bar'.  In this case foo  
is 'C', and bar is 'A'.  Alternately, with 'hg mv foo bar', foo is 'R' and  
bar is 'A'.  The added/removed list would cover the exclusion list below  
in hg, is that not true with git?  Minimally it seems possible to have a  
clean copy source.

Otherwise, LGTM.

> +
> +            if st == '??':
> +                unknown.append(filename1)
> +            elif st == '!!':
> +                ignored.append(filename1)
> +
> +        if opts.get('clean'):
> +            out = self._gitcommand(['ls-files'])
> +            for f in out.split('\n'):
> +                if not f in changedfiles:
> +                    clean.append(f)
>         return scmutil.status(modified, added, removed, deleted,
>                                unknown, ignored, clean)
> diff -r 02d7b5cd373b -r 16c867316afc tests/test-subrepo-git.t
> --- a/tests/test-subrepo-git.t	Tue Feb 10 15:59:12 2015 -0500
> +++ b/tests/test-subrepo-git.t	Mon Mar 09 22:14:09 2015 +0100
> @@ -175,6 +175,8 @@
>    pulling subrepo s from $TESTTMP/gitroot
>    0 files updated, 0 files merged, 0 files removed, 0 files unresolved
>    (branch merge, don't forget to commit)
> +  $ hg st --subrepos s
> +  A s/f
>    $ cat s/f
>    f
>    $ cat s/g
> @@ -944,6 +946,16 @@
>    ? s/c.c
>    ? s/cpp.cpp
>    ? s/foobar.orig
> +  $ hg st --subrepos s --all
> +  A s/.gitignore
> +  ? s/barfoo
> +  ? s/c.c
> +  ? s/cpp.cpp
> +  ? s/foobar.orig
> +  I s/snake.python
> +  C s/f
> +  C s/foobar
> +  C s/g
>    $ hg add --subrepos "glob:**.python"
>    $ hg st --subrepos s
>    A s/.gitignore
> @@ -978,13 +990,41 @@
>    $ hg add s/.gitignore
>    s/.gitignore already tracked!
>    [1]
> +  $ hg add s/g
> +  s/g already tracked!
> +  [1]
> removed files can be re-added
> +removing files using 'rm' or 'git rm' has the same effect,
> +since we ignore the staging area
>    $ hg ci --subrepos -m 'snake'
>    committing subrepository s
>    $ cd s
> +  $ rm snake.python
> +(remove leftover .hg so Mercurial doesn't look for a root here)
> +  $ rm -r .hg
> +  $ hg status --subrepos --all .
> +  R snake.python
> +  ? barfoo
> +  ? c.c
> +  ? cpp.cpp
> +  ? foobar.orig
> +  C .gitignore
> +  C f
> +  C foobar
> +  C g
>    $ git rm snake.python
>    rm 'snake.python'
> +  $ hg status --subrepos --all .
> +  R snake.python
> +  ? barfoo
> +  ? c.c
> +  ? cpp.cpp
> +  ? foobar.orig
> +  C .gitignore
> +  C f
> +  C foobar
> +  C g
>    $ touch snake.python
>    $ cd ..
>    $ hg add s/snake.python
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
Mathias De Maré - March 11, 2015, 8:12 a.m.
On Wed, Mar 11, 2015 at 12:35 AM, Matt Harbison <mharbison72@gmail.com>
wrote:
> On Tue, 10 Mar 2015 14:08:37 -0400, <mathias.demare@gmail.com> wrote:
>
>> # HG changeset patch
>> # User Mathias De Maré <mathias.demare@gmail.com>
>> # Date 1425935649 -3600
>> #      Mon Mar 09 22:14:09 2015 +0100
>> # Node ID 16c867316afce2f57716dbdf93dd5b0760186010
>> # Parent  02d7b5cd373bbb4e8263dad9bfbf9c4c3b0e4e3a
>> subrepo: add status support for ignored and clean files in git subrepos
>>
>> diff -r 02d7b5cd373b -r 16c867316afc mercurial/subrepo.py
>> --- a/mercurial/subrepo.py      Tue Feb 10 15:59:12 2015 -0500
>> +++ b/mercurial/subrepo.py      Mon Mar 09 22:14:09 2015 +0100
>> @@ -1662,13 +1662,42 @@
>>         deleted, unknown, ignored, clean = [], [], [], []
>> +        command = ['status', '--porcelain', '-z']
>>          if opts.get('unknown'):
>> -            command = ['ls-files', '--others', '--exclude-standard']
>> -            out = self._gitcommand(command)
>> -            for line in out.split('\n'):
>> -                if len(line) == 0:
>> -                    continue
>> -                unknown.append(line)
>> +            command += ['--untracked-files=all']
>> +        if opts.get('ignored'):
>> +            command += ['--ignored']
>> +        out = self._gitcommand(command)
>
>
> Small nit: should this command execution be skipped if neither 'ignored'
or
> 'unknown' are set?  (And possibly 'clean', but see below.)
That is indeed a possibility.
Another option could be to replace the earlier 'diff-index' command by this
status command, generating the list of MAD files this way as well.
>
>> +
>> +        changedfiles = set()
>> +        changedfiles.update(modified)
>> +        changedfiles.update(added)
>> +        changedfiles.update(removed)
>> +        for line in out.split('\0'):
>> +            if not line:
>> +                continue
>> +            st = line[0:2]
>> +            #moves and copies show 2 files on one line
>> +            if line.find('\0') >= 0:
>> +                filename1, filename2 = line[3:].split('\0')
>> +            else:
>> +                filename1 = line[3:]
>> +                filename2 = None
>> +
>> +            changedfiles.add(filename1)
>> +            if filename2:
>> +                changedfiles.add(filename2)
>
>
> I don't fully understand this, but it may be my lack of git knowledge.
From
> what I can understand from the git help on status, the file names are the
> source and destination.  But adding both would seem to block both from the
> clean list below.
>
> Consider a clean working dir, and then 'hg cp foo bar'.  In this case foo
is
> 'C', and bar is 'A'.  Alternately, with 'hg mv foo bar', foo is 'R' and
bar
> is 'A'.  The added/removed list would cover the exclusion list below in
hg,
> is that not true with git?  Minimally it seems possible to have a clean
copy
> source.
Looking at 'diff-index', the output is:

> in-place edit  :100644 100644 bcd1234... 0123456... M file0
> copy-edit      :100644 100644 abcd123... 1234567... C68 file1 file2
> rename-edit    :100644 100644 abcd123... 1234567... R86 file1 file3
> create         :000000 100644 0000000... 1234567... A file4
> delete         :100644 000000 1234567... 0000000... D file5
> unmerged       :000000 000000 0000000... 0000000... U file6
>
This means copy+edit and rename+edit are currently not checked by
'diff-index' or 'diff-tree' either, meaning we apparently weren't covering
this correctly earlier either.

I'll put this on my TODO-list for a future patch. Thanks for the remark.

Greetings,
Mathias
>
> Otherwise, LGTM.
>
>> +
>> +            if st == '??':
>> +                unknown.append(filename1)
>> +            elif st == '!!':
>> +                ignored.append(filename1)
>> +
>> +        if opts.get('clean'):
>> +            out = self._gitcommand(['ls-files'])
>> +            for f in out.split('\n'):
>> +                if not f in changedfiles:
>> +                    clean.append(f)
>>         return scmutil.status(modified, added, removed, deleted,
>>                                unknown, ignored, clean)
>> diff -r 02d7b5cd373b -r 16c867316afc tests/test-subrepo-git.t
>> --- a/tests/test-subrepo-git.t  Tue Feb 10 15:59:12 2015 -0500
>> +++ b/tests/test-subrepo-git.t  Mon Mar 09 22:14:09 2015 +0100
>> @@ -175,6 +175,8 @@
>>    pulling subrepo s from $TESTTMP/gitroot
>>    0 files updated, 0 files merged, 0 files removed, 0 files unresolved
>>    (branch merge, don't forget to commit)
>> +  $ hg st --subrepos s
>> +  A s/f
>>    $ cat s/f
>>    f
>>    $ cat s/g
>> @@ -944,6 +946,16 @@
>>    ? s/c.c
>>    ? s/cpp.cpp
>>    ? s/foobar.orig
>> +  $ hg st --subrepos s --all
>> +  A s/.gitignore
>> +  ? s/barfoo
>> +  ? s/c.c
>> +  ? s/cpp.cpp
>> +  ? s/foobar.orig
>> +  I s/snake.python
>> +  C s/f
>> +  C s/foobar
>> +  C s/g
>>    $ hg add --subrepos "glob:**.python"
>>    $ hg st --subrepos s
>>    A s/.gitignore
>> @@ -978,13 +990,41 @@
>>    $ hg add s/.gitignore
>>    s/.gitignore already tracked!
>>    [1]
>> +  $ hg add s/g
>> +  s/g already tracked!
>> +  [1]
>> removed files can be re-added
>> +removing files using 'rm' or 'git rm' has the same effect,
>> +since we ignore the staging area
>>    $ hg ci --subrepos -m 'snake'
>>    committing subrepository s
>>    $ cd s
>> +  $ rm snake.python
>> +(remove leftover .hg so Mercurial doesn't look for a root here)
>> +  $ rm -r .hg
>> +  $ hg status --subrepos --all .
>> +  R snake.python
>> +  ? barfoo
>> +  ? c.c
>> +  ? cpp.cpp
>> +  ? foobar.orig
>> +  C .gitignore
>> +  C f
>> +  C foobar
>> +  C g
>>    $ git rm snake.python
>>    rm 'snake.python'
>> +  $ hg status --subrepos --all .
>> +  R snake.python
>> +  ? barfoo
>> +  ? c.c
>> +  ? cpp.cpp
>> +  ? foobar.orig
>> +  C .gitignore
>> +  C f
>> +  C foobar
>> +  C g
>>    $ touch snake.python
>>    $ cd ..
>>    $ hg add s/snake.python
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel@selenic.com
>> http://selenic.com/mailman/listinfo/mercurial-devel

Patch

diff -r 02d7b5cd373b -r 16c867316afc mercurial/subrepo.py
--- a/mercurial/subrepo.py	Tue Feb 10 15:59:12 2015 -0500
+++ b/mercurial/subrepo.py	Mon Mar 09 22:14:09 2015 +0100
@@ -1662,13 +1662,42 @@ 
 
         deleted, unknown, ignored, clean = [], [], [], []
 
+        command = ['status', '--porcelain', '-z']
         if opts.get('unknown'):
-            command = ['ls-files', '--others', '--exclude-standard']
-            out = self._gitcommand(command)
-            for line in out.split('\n'):
-                if len(line) == 0:
-                    continue
-                unknown.append(line)
+            command += ['--untracked-files=all']
+        if opts.get('ignored'):
+            command += ['--ignored']
+        out = self._gitcommand(command)
+
+        changedfiles = set()
+        changedfiles.update(modified)
+        changedfiles.update(added)
+        changedfiles.update(removed)
+        for line in out.split('\0'):
+            if not line:
+                continue
+            st = line[0:2]
+            #moves and copies show 2 files on one line
+            if line.find('\0') >= 0:
+                filename1, filename2 = line[3:].split('\0')
+            else:
+                filename1 = line[3:]
+                filename2 = None
+
+            changedfiles.add(filename1)
+            if filename2:
+                changedfiles.add(filename2)
+
+            if st == '??':
+                unknown.append(filename1)
+            elif st == '!!':
+                ignored.append(filename1)
+
+        if opts.get('clean'):
+            out = self._gitcommand(['ls-files'])
+            for f in out.split('\n'):
+                if not f in changedfiles:
+                    clean.append(f)
 
         return scmutil.status(modified, added, removed, deleted,
                               unknown, ignored, clean)
diff -r 02d7b5cd373b -r 16c867316afc tests/test-subrepo-git.t
--- a/tests/test-subrepo-git.t	Tue Feb 10 15:59:12 2015 -0500
+++ b/tests/test-subrepo-git.t	Mon Mar 09 22:14:09 2015 +0100
@@ -175,6 +175,8 @@ 
   pulling subrepo s from $TESTTMP/gitroot
   0 files updated, 0 files merged, 0 files removed, 0 files unresolved
   (branch merge, don't forget to commit)
+  $ hg st --subrepos s
+  A s/f
   $ cat s/f
   f
   $ cat s/g
@@ -944,6 +946,16 @@ 
   ? s/c.c
   ? s/cpp.cpp
   ? s/foobar.orig
+  $ hg st --subrepos s --all
+  A s/.gitignore
+  ? s/barfoo
+  ? s/c.c
+  ? s/cpp.cpp
+  ? s/foobar.orig
+  I s/snake.python
+  C s/f
+  C s/foobar
+  C s/g
   $ hg add --subrepos "glob:**.python"
   $ hg st --subrepos s
   A s/.gitignore
@@ -978,13 +990,41 @@ 
   $ hg add s/.gitignore
   s/.gitignore already tracked!
   [1]
+  $ hg add s/g
+  s/g already tracked!
+  [1]
 
 removed files can be re-added
+removing files using 'rm' or 'git rm' has the same effect,
+since we ignore the staging area
   $ hg ci --subrepos -m 'snake'
   committing subrepository s
   $ cd s
+  $ rm snake.python
+(remove leftover .hg so Mercurial doesn't look for a root here)
+  $ rm -r .hg
+  $ hg status --subrepos --all .
+  R snake.python
+  ? barfoo
+  ? c.c
+  ? cpp.cpp
+  ? foobar.orig
+  C .gitignore
+  C f
+  C foobar
+  C g
   $ git rm snake.python
   rm 'snake.python'
+  $ hg status --subrepos --all .
+  R snake.python
+  ? barfoo
+  ? c.c
+  ? cpp.cpp
+  ? foobar.orig
+  C .gitignore
+  C f
+  C foobar
+  C g
   $ touch snake.python
   $ cd ..
   $ hg add s/snake.python