Patchwork largefiles: set the extension as enabled locally after a clone requiring it

login
register
mail settings
Submitter Matt Harbison
Date Feb. 3, 2015, 5:13 p.m.
Message ID <fda61f818913f4afb4ad.1422983602@waste.org>
Download mbox | patch
Permalink /patch/7635/
State Accepted
Commit e1dbe0b215ae137eec53ceb12440536d570a83d2
Headers show

Comments

Matt Harbison - Feb. 3, 2015, 5:13 p.m.
# HG changeset patch
# User Matt Harbison <matt_harbison@yahoo.com>
# Date 1422925121 18000
#      Mon Feb 02 19:58:41 2015 -0500
# Node ID fda61f818913f4afb4adf5704eba990fe2ac29e5
# Parent  8b88870cbd1eeefaee0af053ae36728f8c0a1847
largefiles: set the extension as enabled locally after a clone requiring it

When cloning a repo that requires largefiles, the user had to either enable the
extension on the command line and then manually edit the local hgrc file after
the clone, or just enable it globally for the user.  Since it is a feature of
last resort, and materially affects even repos without any largefiles when it is
enabled, we should make it easier to not have it enabled globally.

This simply adds the enabling statement to the local hgrc if the requires file
mandates its use (which only happens after the first largefile is committed).
That means that a user who works with a mix of largefile and normal repos can
always clone with '--config extensions.largefiles=', and the extension is
permanently enabled or not as appropriate.

The change in test-largefiles.t is simply because the order of loading rebase
and largefiles changed.  The same change occurs if the order is flipped in the
hgrc file at the top of the test.
Augie Fackler - Feb. 3, 2015, 8:09 p.m.
On Tue, Feb 03, 2015 at 11:13:22AM -0600, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison <matt_harbison@yahoo.com>
> # Date 1422925121 18000
> #      Mon Feb 02 19:58:41 2015 -0500
> # Node ID fda61f818913f4afb4adf5704eba990fe2ac29e5
> # Parent  8b88870cbd1eeefaee0af053ae36728f8c0a1847
> largefiles: set the extension as enabled locally after a clone requiring it

This looks reasonable to me.

>
> When cloning a repo that requires largefiles, the user had to either enable the
> extension on the command line and then manually edit the local hgrc file after
> the clone, or just enable it globally for the user.  Since it is a feature of
> last resort, and materially affects even repos without any largefiles when it is
> enabled, we should make it easier to not have it enabled globally.
>
> This simply adds the enabling statement to the local hgrc if the requires file
> mandates its use (which only happens after the first largefile is committed).
> That means that a user who works with a mix of largefile and normal repos can
> always clone with '--config extensions.largefiles=', and the extension is
> permanently enabled or not as appropriate.
>
> The change in test-largefiles.t is simply because the order of loading rebase
> and largefiles changed.  The same change occurs if the order is flipped in the
> hgrc file at the top of the test.
>
> diff --git a/hgext/largefiles/overrides.py b/hgext/largefiles/overrides.py
> --- a/hgext/largefiles/overrides.py
> +++ b/hgext/largefiles/overrides.py
> @@ -820,6 +820,14 @@
>          sourcerepo, destrepo = result
>          repo = destrepo.local()
>
> +        # If largefiles is required for this repo, permanently enable it locally
> +        if 'largefiles' in repo.requirements:
> +            fp = repo.vfs('hgrc', 'a', text=True)
> +            try:
> +                fp.write('\n[extensions]\nlargefiles=\n')
> +            finally:
> +                fp.close()
> +
>          # Caching is implicitly limited to 'rev' option, since the dest repo was
>          # truncated at that point.  The user may expect a download count with
>          # this option, so attempt whether or not this is a largefile repo.
> diff --git a/tests/test-largefiles.t b/tests/test-largefiles.t
> --- a/tests/test-largefiles.t
> +++ b/tests/test-largefiles.t
> @@ -1183,12 +1183,12 @@
>    adding manifests
>    adding file changes
>    added 1 changesets with 2 changes to 2 files (+1 heads)
> -  0 largefiles cached
>    rebasing 8:f574fb32bb45 "modify normal file largefile in repo d"
>    Invoking status precommit hook
>    M sub/normal4
>    M sub2/large6
>    saved backup bundle to $TESTTMP/d/.hg/strip-backup/f574fb32bb45-dd1d9f80-backup.hg (glob)
> +  0 largefiles cached
>    $ [ -f .hg/largefiles/e166e74c7303192238d60af5a9c4ce9bef0b7928 ]
>    $ hg log --template '{rev}:{node|short}  {desc|firstline}\n'
>    9:598410d3eb9a  modify normal file largefile in repo d
> diff --git a/tests/test-subrepo-deep-nested-change.t b/tests/test-subrepo-deep-nested-change.t
> --- a/tests/test-subrepo-deep-nested-change.t
> +++ b/tests/test-subrepo-deep-nested-change.t
> @@ -46,12 +46,29 @@
>
>  Clone main
>
> -  $ hg clone main cloned
> +  $ hg --config extensions.largefiles= clone main cloned
>    updating to branch default
>    cloning subrepo sub1 from $TESTTMP/sub1
>    cloning subrepo sub1/sub2 from $TESTTMP/sub2 (glob)
>    3 files updated, 0 files merged, 0 files removed, 0 files unresolved
>
> +Largefiles is NOT enabled in the clone if the source repo doesn't require it
> +  $ cat cloned/.hg/hgrc
> +  # example repository config (see "hg help config" for more info)
> +  [paths]
> +  default = $TESTTMP/main (glob)
> +
> +  # path aliases to other clones of this repo in URLs or filesystem paths
> +  # (see "hg help config.paths" for more info)
> +  #
> +  # default-push = ssh://jdoe@example.net/hg/jdoes-fork
> +  # my-fork      = ssh://jdoe@example.net/hg/jdoes-fork
> +  # my-clone     = /home/jdoe/jdoes-clone
> +
> +  [ui]
> +  # name and email (local to this repository, optional), e.g.
> +  # username = Jane Doe <jdoe@example.com>
> +
>  Checking cloned repo ids
>
>    $ printf "cloned " ; hg id -R cloned
> @@ -319,6 +336,31 @@
>    ../archive_lf/sub1/sub2/large.bin
>    $ rm -rf ../archive_lf
>
> +The local repo enables largefiles if a largefiles repo is cloned
> +  $ hg showconfig extensions
> +  abort: repository requires features unknown to this Mercurial: largefiles!
> +  (see http://mercurial.selenic.com/wiki/MissingRequirement for more information)
> +  [255]
> +  $ hg --config extensions.largefiles= clone -qU . ../lfclone
> +  $ cat ../lfclone/.hg/hgrc
> +  # example repository config (see "hg help config" for more info)
> +  [paths]
> +  default = $TESTTMP/cloned (glob)
> +
> +  # path aliases to other clones of this repo in URLs or filesystem paths
> +  # (see "hg help config.paths" for more info)
> +  #
> +  # default-push = ssh://jdoe@example.net/hg/jdoes-fork
> +  # my-fork      = ssh://jdoe@example.net/hg/jdoes-fork
> +  # my-clone     = /home/jdoe/jdoes-clone
> +
> +  [ui]
> +  # name and email (local to this repository, optional), e.g.
> +  # username = Jane Doe <jdoe@example.com>
> +
> +  [extensions]
> +  largefiles=
> +
>  Find an exact match to a standin (should archive nothing)
>    $ hg --config extensions.largefiles= archive -S -I 'sub/sub2/.hglf/large.bin' ../archive_lf
>    $ find ../archive_lf 2> /dev/null | sort
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
Matt Harbison - Feb. 4, 2015, 12:30 a.m.
On Tue, 03 Feb 2015 15:09:34 -0500, Augie Fackler <raf@durin42.com> wrote:

> On Tue, Feb 03, 2015 at 11:13:22AM -0600, Matt Harbison wrote:
>> # HG changeset patch
>> # User Matt Harbison <matt_harbison@yahoo.com>
>> # Date 1422925121 18000
>> #      Mon Feb 02 19:58:41 2015 -0500
>> # Node ID fda61f818913f4afb4adf5704eba990fe2ac29e5
>> # Parent  8b88870cbd1eeefaee0af053ae36728f8c0a1847
>> largefiles: set the extension as enabled locally after a clone  
>> requiring it
>
> This looks reasonable to me.

As an aside, I posted this using the experimental pushgate at around noon  
EST.  The first push aborted with an internal server error 500.  I tried  
5-10 minutes later with --traceback on the off chance it would yield  
anything useful, but it went through this time, so IDK what the problem  
was.

Random thoughts:

- It seemed to take a surprisingly long period of time to push, but I  
haven't pushed to a repo over the internet before.  So I'm not sure how  
much was the internet connection, the actual push, and whatever checking  
(I assume) it is doing.  I know the remote output is only displayed after  
the local output with a Windows push, so maybe I missed it, but maybe a  
"This will take several minutes" message would help?

- I don't have the message that was output in front of me, but I remember  
the last line was 'abort/transaction rollback'.  After I checked the ML  
and saw it made it, I understood what was going on, but maybe that can be  
hidden somehow?

Otherwise, it's a pretty neat idea.

--Matt
Augie Fackler - Feb. 4, 2015, 12:37 a.m.
On Feb 3, 2015, at 7:30 PM, Matt Harbison <mharbison72@gmail.com> wrote:

> On Tue, 03 Feb 2015 15:09:34 -0500, Augie Fackler <raf@durin42.com> wrote:
> 
>> On Tue, Feb 03, 2015 at 11:13:22AM -0600, Matt Harbison wrote:
>>> # HG changeset patch
>>> # User Matt Harbison <matt_harbison@yahoo.com>
>>> # Date 1422925121 18000
>>> #      Mon Feb 02 19:58:41 2015 -0500
>>> # Node ID fda61f818913f4afb4adf5704eba990fe2ac29e5
>>> # Parent  8b88870cbd1eeefaee0af053ae36728f8c0a1847
>>> largefiles: set the extension as enabled locally after a clone requiring it
>> 
>> This looks reasonable to me.
> 
> As an aside, I posted this using the experimental pushgate at around noon EST.  The first push aborted with an internal server error 500.  I tried 5-10 minutes later with --traceback on the off chance it would yield anything useful, but it went through this time, so IDK what the problem was.
> 
> Random thoughts:
> 
> - It seemed to take a surprisingly long period of time to push, but I haven't pushed to a repo over the internet before.  So I'm not sure how much was the internet connection, the actual push, and whatever checking (I assume) it is doing.  I know the remote output is only displayed after the local output with a Windows push, so maybe I missed it, but maybe a "This will take several minutes" message would help?
> 
> - I don't have the message that was output in front of me, but I remember the last line was 'abort/transaction rollback'.  After I checked the ML and saw it made it, I understood what was going on, but maybe that can be hidden somehow?
> 
> Otherwise, it's a pretty neat idea.

Aha! That explains why your message flunked spf and ended up in spam.

We probably should have it do something like “pushbot on behalf of foo <pushbot@selenic.com>” and cc the author of the change.

> 
> --Matt
Matt Harbison - Feb. 4, 2015, 12:43 a.m.
On Tue, 03 Feb 2015 19:37:56 -0500, Augie Fackler <raf@durin42.com> wrote:

>
> On Feb 3, 2015, at 7:30 PM, Matt Harbison <mharbison72@gmail.com> wrote:
>
>> On Tue, 03 Feb 2015 15:09:34 -0500, Augie Fackler <raf@durin42.com>  
>> wrote:
>>
>>> On Tue, Feb 03, 2015 at 11:13:22AM -0600, Matt Harbison wrote:
>>>> # HG changeset patch
>>>> # User Matt Harbison <matt_harbison@yahoo.com>
>>>> # Date 1422925121 18000
>>>> #      Mon Feb 02 19:58:41 2015 -0500
>>>> # Node ID fda61f818913f4afb4adf5704eba990fe2ac29e5
>>>> # Parent  8b88870cbd1eeefaee0af053ae36728f8c0a1847
>>>> largefiles: set the extension as enabled locally after a clone  
>>>> requiring it
>>>
>>> This looks reasonable to me.
>>
>> As an aside, I posted this using the experimental pushgate at around  
>> noon EST.  The first push aborted with an internal server error 500.  I  
>> tried 5-10 minutes later with --traceback on the off chance it would  
>> yield anything useful, but it went through this time, so IDK what the  
>> problem was.
>>
>> Random thoughts:
>>
>> - It seemed to take a surprisingly long period of time to push, but I  
>> haven't pushed to a repo over the internet before.  So I'm not sure how  
>> much was the internet connection, the actual push, and whatever  
>> checking (I assume) it is doing.  I know the remote output is only  
>> displayed after the local output with a Windows push, so maybe I missed  
>> it, but maybe a "This will take several minutes" message would help?
>>
>> - I don't have the message that was output in front of me, but I  
>> remember the last line was 'abort/transaction rollback'.  After I  
>> checked the ML and saw it made it, I understood what was going on, but  
>> maybe that can be hidden somehow?
>>
>> Otherwise, it's a pretty neat idea.
>
> Aha! That explains why your message flunked spf and ended up in spam.
>
> We probably should have it do something like “pushbot on behalf of foo  
> <pushbot@selenic.com>” and cc the author of the change.
>

Maybe it's because of the yahoo address in the author line?  I've been  
patchbombing using gmail because of spam issues, but kept the author line  
for consistency, and because my phone won't log into gmail.  I like the cc  
idea too.

--Matt
Matt Mackall - Feb. 4, 2015, 12:52 a.m.
On Tue, 2015-02-03 at 19:37 -0500, Augie Fackler wrote:
> On Feb 3, 2015, at 7:30 PM, Matt Harbison <mharbison72@gmail.com> wrote:
> 
> > On Tue, 03 Feb 2015 15:09:34 -0500, Augie Fackler <raf@durin42.com> wrote:
> > 
> >> On Tue, Feb 03, 2015 at 11:13:22AM -0600, Matt Harbison wrote:
> >>> # HG changeset patch
> >>> # User Matt Harbison <matt_harbison@yahoo.com>
> >>> # Date 1422925121 18000
> >>> #      Mon Feb 02 19:58:41 2015 -0500
> >>> # Node ID fda61f818913f4afb4adf5704eba990fe2ac29e5
> >>> # Parent  8b88870cbd1eeefaee0af053ae36728f8c0a1847
> >>> largefiles: set the extension as enabled locally after a clone requiring it
> >> 
> >> This looks reasonable to me.
> > 
> > As an aside, I posted this using the experimental pushgate at around
> noon EST.  The first push aborted with an internal server error 500.

Huh. No traceback logged here.

>   I tried 5-10 minutes later with --traceback on the off chance it
> would yield anything useful, but it went through this time, so IDK
> what the problem was.
> > 
> > Random thoughts:
> > 
> > - It seemed to take a surprisingly long period of time to push, but
> I haven't pushed to a repo over the internet before.  So I'm not sure
> how much was the internet connection, the actual push, and whatever
> checking (I assume) it is doing.  I know the remote output is only
> displayed after the local output with a Windows push, so maybe I
> missed it, but maybe a "This will take several minutes" message would
> help?

Should take well under a minute for a single patch.. but the server load
can occasionally be significant.

> Aha! That explains why your message flunked spf and ended up in spam.
> 
> We probably should have it do something like “pushbot on behalf of foo <pushbot@selenic.com>” and cc the author of the change.

Yep.
Matt Mackall - Feb. 4, 2015, 1:17 a.m.
On Tue, 2015-02-03 at 11:13 -0600, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison <matt_harbison@yahoo.com>
> # Date 1422925121 18000
> #      Mon Feb 02 19:58:41 2015 -0500
> # Node ID fda61f818913f4afb4adf5704eba990fe2ac29e5
> # Parent  8b88870cbd1eeefaee0af053ae36728f8c0a1847
> largefiles: set the extension as enabled locally after a clone requiring it

Sure, queued for default.

Patch

diff --git a/hgext/largefiles/overrides.py b/hgext/largefiles/overrides.py
--- a/hgext/largefiles/overrides.py
+++ b/hgext/largefiles/overrides.py
@@ -820,6 +820,14 @@ 
         sourcerepo, destrepo = result
         repo = destrepo.local()
 
+        # If largefiles is required for this repo, permanently enable it locally
+        if 'largefiles' in repo.requirements:
+            fp = repo.vfs('hgrc', 'a', text=True)
+            try:
+                fp.write('\n[extensions]\nlargefiles=\n')
+            finally:
+                fp.close()
+
         # Caching is implicitly limited to 'rev' option, since the dest repo was
         # truncated at that point.  The user may expect a download count with
         # this option, so attempt whether or not this is a largefile repo.
diff --git a/tests/test-largefiles.t b/tests/test-largefiles.t
--- a/tests/test-largefiles.t
+++ b/tests/test-largefiles.t
@@ -1183,12 +1183,12 @@ 
   adding manifests
   adding file changes
   added 1 changesets with 2 changes to 2 files (+1 heads)
-  0 largefiles cached
   rebasing 8:f574fb32bb45 "modify normal file largefile in repo d"
   Invoking status precommit hook
   M sub/normal4
   M sub2/large6
   saved backup bundle to $TESTTMP/d/.hg/strip-backup/f574fb32bb45-dd1d9f80-backup.hg (glob)
+  0 largefiles cached
   $ [ -f .hg/largefiles/e166e74c7303192238d60af5a9c4ce9bef0b7928 ]
   $ hg log --template '{rev}:{node|short}  {desc|firstline}\n'
   9:598410d3eb9a  modify normal file largefile in repo d
diff --git a/tests/test-subrepo-deep-nested-change.t b/tests/test-subrepo-deep-nested-change.t
--- a/tests/test-subrepo-deep-nested-change.t
+++ b/tests/test-subrepo-deep-nested-change.t
@@ -46,12 +46,29 @@ 
 
 Clone main
 
-  $ hg clone main cloned
+  $ hg --config extensions.largefiles= clone main cloned
   updating to branch default
   cloning subrepo sub1 from $TESTTMP/sub1
   cloning subrepo sub1/sub2 from $TESTTMP/sub2 (glob)
   3 files updated, 0 files merged, 0 files removed, 0 files unresolved
 
+Largefiles is NOT enabled in the clone if the source repo doesn't require it
+  $ cat cloned/.hg/hgrc
+  # example repository config (see "hg help config" for more info)
+  [paths]
+  default = $TESTTMP/main (glob)
+  
+  # path aliases to other clones of this repo in URLs or filesystem paths
+  # (see "hg help config.paths" for more info)
+  #
+  # default-push = ssh://jdoe@example.net/hg/jdoes-fork
+  # my-fork      = ssh://jdoe@example.net/hg/jdoes-fork
+  # my-clone     = /home/jdoe/jdoes-clone
+  
+  [ui]
+  # name and email (local to this repository, optional), e.g.
+  # username = Jane Doe <jdoe@example.com>
+
 Checking cloned repo ids
 
   $ printf "cloned " ; hg id -R cloned
@@ -319,6 +336,31 @@ 
   ../archive_lf/sub1/sub2/large.bin
   $ rm -rf ../archive_lf
 
+The local repo enables largefiles if a largefiles repo is cloned
+  $ hg showconfig extensions
+  abort: repository requires features unknown to this Mercurial: largefiles!
+  (see http://mercurial.selenic.com/wiki/MissingRequirement for more information)
+  [255]
+  $ hg --config extensions.largefiles= clone -qU . ../lfclone
+  $ cat ../lfclone/.hg/hgrc
+  # example repository config (see "hg help config" for more info)
+  [paths]
+  default = $TESTTMP/cloned (glob)
+  
+  # path aliases to other clones of this repo in URLs or filesystem paths
+  # (see "hg help config.paths" for more info)
+  #
+  # default-push = ssh://jdoe@example.net/hg/jdoes-fork
+  # my-fork      = ssh://jdoe@example.net/hg/jdoes-fork
+  # my-clone     = /home/jdoe/jdoes-clone
+  
+  [ui]
+  # name and email (local to this repository, optional), e.g.
+  # username = Jane Doe <jdoe@example.com>
+  
+  [extensions]
+  largefiles=
+
 Find an exact match to a standin (should archive nothing)
   $ hg --config extensions.largefiles= archive -S -I 'sub/sub2/.hglf/large.bin' ../archive_lf
   $ find ../archive_lf 2> /dev/null | sort