Patchwork [03,of,10] commands: stub for debugupgraderepo command

login
register
mail settings
Submitter Gregory Szorc
Date Nov. 6, 2016, 4:40 a.m.
Message ID <9daec9c7adabe8c84cf2.1478407219@ubuntu-vm-main>
Download mbox | patch
Permalink /patch/17365/
State Changes Requested
Headers show

Comments

Gregory Szorc - Nov. 6, 2016, 4:40 a.m.
# HG changeset patch
# User Gregory Szorc <gregory.szorc@gmail.com>
# Date 1478391613 25200
#      Sat Nov 05 17:20:13 2016 -0700
# Node ID 9daec9c7adabe8c84cf2c01fc938e010ee4884d6
# Parent  ed3241d8b00e476818ff1aec3db0136bf960de35
commands: stub for debugupgraderepo command

Currently, if Mercurial introduces a new repository/store feature or
changes behavior of an existing feature, users must perform an
`hg clone` to create a new repository with hopefully the
correct/optimal settings. Unfortunately, even `hg clone` may not
give the correct results. For example, if you do a local `hg clone`,
you may get hardlinks to revlog files that inherit the old state.
If you `hg clone` from a remote or `hg clone --pull`, changegroup
application may bypass some optimization, such as converting to
generaldelta.

Optimizing a repository is harder than it seems and requires more
than a simple `hg` command invocation.

This patch starts the process of changing that. We introduce
`hg debugupgraderepo`, a command that performs an in-place upgrade
of a repository to use new, optimal features. The command is just
a stub right now. Features will be added in subsequent patches.
Pierre-Yves David - Nov. 22, 2016, 2:05 a.m.
On 11/06/2016 05:40 AM, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1478391613 25200
> #      Sat Nov 05 17:20:13 2016 -0700
> # Node ID 9daec9c7adabe8c84cf2c01fc938e010ee4884d6
> # Parent  ed3241d8b00e476818ff1aec3db0136bf960de35
> commands: stub for debugupgraderepo command
>
> Currently, if Mercurial introduces a new repository/store feature or
> changes behavior of an existing feature, users must perform an
> `hg clone` to create a new repository with hopefully the
> correct/optimal settings. Unfortunately, even `hg clone` may not
> give the correct results. For example, if you do a local `hg clone`,
> you may get hardlinks to revlog files that inherit the old state.
> If you `hg clone` from a remote or `hg clone --pull`, changegroup
> application may bypass some optimization, such as converting to
> generaldelta.
>
> Optimizing a repository is harder than it seems and requires more
> than a simple `hg` command invocation.
>
> This patch starts the process of changing that. We introduce
> `hg debugupgraderepo`, a command that performs an in-place upgrade
> of a repository to use new, optimal features. The command is just
> a stub right now. Features will be added in subsequent patches.

I had a similar series in progress which a slightly different 
naming/behavior.

* 'hg debugformat' list details about the current repository format (and 
possible upgrade)

* 'hg debugformat --upgrade' performs actual upgrade (more on this in 
the next patch)

I'm not saying you should restart your whole series to match this, but 
I'll most probably submit a rename proposal to match the above once this 
is in.

> diff --git a/mercurial/commands.py b/mercurial/commands.py
> --- a/mercurial/commands.py
> +++ b/mercurial/commands.py

Since the first bits or your series are in, you should consider adding 
the command to the "mercurial/debugcommands.py" ;-)

> @@ -3747,6 +3747,17 @@ def debugtemplate(ui, repo, tmpl, **opts
>              displayer.show(repo[r], **props)
>          displayer.close()
>
> +@command('debugupgraderepo', dryrunopts)
> +def debugupgraderepo(ui, repo, **opts):
> +    """upgrade a repository to use different features
> +
> +    During the upgrade, the repository will be locked and no writes will be
> +    allowed.
> +
> +    At times during the upgrade, the repository may not be readable.
> +    """
> +    raise error.Abort(_('not yet implemented'))
> +
>  @command('debugwalk', walkopts, _('[OPTION]... [FILE]...'), inferrepo=True)
>  def debugwalk(ui, repo, *pats, **opts):
>      """show how files match on given patterns"""
> diff --git a/tests/test-completion.t b/tests/test-completion.t
> --- a/tests/test-completion.t
> +++ b/tests/test-completion.t
> @@ -109,6 +109,7 @@ Show debug commands if there are no othe
>    debugsub
>    debugsuccessorssets
>    debugtemplate
> +  debugupgraderepo
>    debugwalk
>    debugwireargs
>
> @@ -274,6 +275,7 @@ Show all commands + options
>    debugsub: rev
>    debugsuccessorssets:
>    debugtemplate: rev, define
> +  debugupgraderepo: dry-run
>    debugwalk: include, exclude
>    debugwireargs: three, four, five, ssh, remotecmd, insecure
>    files: rev, print0, include, exclude, template, subrepos
> diff --git a/tests/test-help.t b/tests/test-help.t
> --- a/tests/test-help.t
> +++ b/tests/test-help.t
> @@ -917,6 +917,8 @@ Test list of internal help commands
>                   show set of successors for revision
>     debugtemplate
>                   parse and apply a template
> +   debugupgraderepo
> +                 upgrade a repository to use different features
>     debugwalk     show how files match on given patterns
>     debugwireargs
>                   (no help text available)
> diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
> new file mode 100644
> --- /dev/null
> +++ b/tests/test-upgrade-repo.t
> @@ -0,0 +1,5 @@
> +  $ hg init empty
> +  $ cd empty
> +  $ hg debugupgraderepo
> +  abort: not yet implemented
> +  [255]
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
Gregory Szorc - Nov. 22, 2016, 5:15 a.m.
On Mon, Nov 21, 2016 at 6:05 PM, Pierre-Yves David <
pierre-yves.david@ens-lyon.org> wrote:

>
>
> On 11/06/2016 05:40 AM, Gregory Szorc wrote:
>
>> # HG changeset patch
>> # User Gregory Szorc <gregory.szorc@gmail.com>
>> # Date 1478391613 25200
>> #      Sat Nov 05 17:20:13 2016 -0700
>> # Node ID 9daec9c7adabe8c84cf2c01fc938e010ee4884d6
>> # Parent  ed3241d8b00e476818ff1aec3db0136bf960de35
>> commands: stub for debugupgraderepo command
>>
>> Currently, if Mercurial introduces a new repository/store feature or
>> changes behavior of an existing feature, users must perform an
>> `hg clone` to create a new repository with hopefully the
>> correct/optimal settings. Unfortunately, even `hg clone` may not
>> give the correct results. For example, if you do a local `hg clone`,
>> you may get hardlinks to revlog files that inherit the old state.
>> If you `hg clone` from a remote or `hg clone --pull`, changegroup
>> application may bypass some optimization, such as converting to
>> generaldelta.
>>
>> Optimizing a repository is harder than it seems and requires more
>> than a simple `hg` command invocation.
>>
>> This patch starts the process of changing that. We introduce
>> `hg debugupgraderepo`, a command that performs an in-place upgrade
>> of a repository to use new, optimal features. The command is just
>> a stub right now. Features will be added in subsequent patches.
>>
>
> I had a similar series in progress which a slightly different
> naming/behavior.
>
> * 'hg debugformat' list details about the current repository format (and
> possible upgrade)
>
> * 'hg debugformat --upgrade' performs actual upgrade (more on this in the
> next patch)
>
> I'm not saying you should restart your whole series to match this, but
> I'll most probably submit a rename proposal to match the above once this is
> in.
>

TBH, I'm not a fan of "debugformat." The main reason is "format" isn't
exactly unambiguous: it can mean several things from deleting all data
(i.e. "formatting a disk") to UI presentation options. If I weren't aware
that the [format] config section existed, I'd have no clue that "format"
referred to the on-disk repository "format."

I think having "upgrade" in the name is beneficial because it says what the
command does ("upgrade" the repository to a new and more modern format).
That being said, I would be open to the idea of naming it "debugreformat."
That "re" there is important: it tells a user not familiar with the
"format" terminology that it changes something.

Another thing I like about your proposal is that `hg debugformat` is
read-only by default. While I typically don't like commands that are both
read-only and perform modifications, I think "upgrade/format" is special
enough that it really should require positive affirmation / consent before
doing anything. So I think I'll change the behavior in V2 to print possible
upgrades by default and only take action if a flag is specified. I'll have
to think a bit more about this...


> diff --git a/mercurial/commands.py b/mercurial/commands.py
>> --- a/mercurial/commands.py
>> +++ b/mercurial/commands.py
>>
>
> Since the first bits or your series are in, you should consider adding the
> command to the "mercurial/debugcommands.py" ;-)
>

Will do. I think I started authoring this series before debugcommands.py
existed!


>
> @@ -3747,6 +3747,17 @@ def debugtemplate(ui, repo, tmpl, **opts
>>              displayer.show(repo[r], **props)
>>          displayer.close()
>>
>> +@command('debugupgraderepo', dryrunopts)
>> +def debugupgraderepo(ui, repo, **opts):
>> +    """upgrade a repository to use different features
>> +
>> +    During the upgrade, the repository will be locked and no writes will
>> be
>> +    allowed.
>> +
>> +    At times during the upgrade, the repository may not be readable.
>> +    """
>> +    raise error.Abort(_('not yet implemented'))
>> +
>>  @command('debugwalk', walkopts, _('[OPTION]... [FILE]...'),
>> inferrepo=True)
>>  def debugwalk(ui, repo, *pats, **opts):
>>      """show how files match on given patterns"""
>> diff --git a/tests/test-completion.t b/tests/test-completion.t
>> --- a/tests/test-completion.t
>> +++ b/tests/test-completion.t
>> @@ -109,6 +109,7 @@ Show debug commands if there are no othe
>>    debugsub
>>    debugsuccessorssets
>>    debugtemplate
>> +  debugupgraderepo
>>    debugwalk
>>    debugwireargs
>>
>> @@ -274,6 +275,7 @@ Show all commands + options
>>    debugsub: rev
>>    debugsuccessorssets:
>>    debugtemplate: rev, define
>> +  debugupgraderepo: dry-run
>>    debugwalk: include, exclude
>>    debugwireargs: three, four, five, ssh, remotecmd, insecure
>>    files: rev, print0, include, exclude, template, subrepos
>> diff --git a/tests/test-help.t b/tests/test-help.t
>> --- a/tests/test-help.t
>> +++ b/tests/test-help.t
>> @@ -917,6 +917,8 @@ Test list of internal help commands
>>                   show set of successors for revision
>>     debugtemplate
>>                   parse and apply a template
>> +   debugupgraderepo
>> +                 upgrade a repository to use different features
>>     debugwalk     show how files match on given patterns
>>     debugwireargs
>>                   (no help text available)
>> diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
>> new file mode 100644
>> --- /dev/null
>> +++ b/tests/test-upgrade-repo.t
>> @@ -0,0 +1,5 @@
>> +  $ hg init empty
>> +  $ cd empty
>> +  $ hg debugupgraderepo
>> +  abort: not yet implemented
>> +  [255]
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel@mercurial-scm.org
>> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>>
>>
> --
> Pierre-Yves David
>
Pierre-Yves David - Nov. 23, 2016, 4:24 p.m.
On 11/22/2016 06:15 AM, Gregory Szorc wrote:
>
>
> On Mon, Nov 21, 2016 at 6:05 PM, Pierre-Yves David
> <pierre-yves.david@ens-lyon.org <mailto:pierre-yves.david@ens-lyon.org>>
> wrote:
>
>
>
>     On 11/06/2016 05:40 AM, Gregory Szorc wrote:
>
>         # HG changeset patch
>         # User Gregory Szorc <gregory.szorc@gmail.com
>         <mailto:gregory.szorc@gmail.com>>
>         # Date 1478391613 25200
>         #      Sat Nov 05 17:20:13 2016 -0700
>         # Node ID 9daec9c7adabe8c84cf2c01fc938e010ee4884d6
>         # Parent  ed3241d8b00e476818ff1aec3db0136bf960de35
>         commands: stub for debugupgraderepo command
>
>         Currently, if Mercurial introduces a new repository/store feature or
>         changes behavior of an existing feature, users must perform an
>         `hg clone` to create a new repository with hopefully the
>         correct/optimal settings. Unfortunately, even `hg clone` may not
>         give the correct results. For example, if you do a local `hg clone`,
>         you may get hardlinks to revlog files that inherit the old state.
>         If you `hg clone` from a remote or `hg clone --pull`, changegroup
>         application may bypass some optimization, such as converting to
>         generaldelta.
>
>         Optimizing a repository is harder than it seems and requires more
>         than a simple `hg` command invocation.
>
>         This patch starts the process of changing that. We introduce
>         `hg debugupgraderepo`, a command that performs an in-place upgrade
>         of a repository to use new, optimal features. The command is just
>         a stub right now. Features will be added in subsequent patches.
>
>
>     I had a similar series in progress which a slightly different
>     naming/behavior.
>
>     * 'hg debugformat' list details about the current repository format
>     (and possible upgrade)
>
>     * 'hg debugformat --upgrade' performs actual upgrade (more on this
>     in the next patch)
>
>     I'm not saying you should restart your whole series to match this,
>     but I'll most probably submit a rename proposal to match the above
>     once this is in.
>
>
> TBH, I'm not a fan of "debugformat." The main reason is "format" isn't
> exactly unambiguous: it can mean several things from deleting all data
> (i.e. "formatting a disk") to UI presentation options. If I weren't
> aware that the [format] config section existed, I'd have no clue that
> "format" referred to the on-disk repository "format."

As I know the "format" config for a long time, the "debugformat" name 
was natural to me. I'm not sure how the naming is important, user are 
not really expected to go hunting for debug command themselve, and if 
they do a proper help is probably enough. I won't have a too strong 
opinion about the name but I see a good feedback cycle between 
"debugformat" and "[format]" knowing/finding one help to find the other.

> I think having "upgrade" in the name is beneficial because it says what
> the command does ("upgrade" the repository to a new and more modern
> format). That being said, I would be open to the idea of naming it
> "debugreformat." That "re" there is important: it tells a user not
> familiar with the "format" terminology that it changes something.

I would expect mentioning "upgrade" in the first line of the help (shown 
by `hg help debug`) to be enough. "reformat" reads a bit strange to me 
but that could be a good middle ground. (Also, I'm not a native speaker)

> Another thing I like about your proposal is that `hg debugformat` is
> read-only by default. While I typically don't like commands that are
> both read-only and perform modifications, I think "upgrade/format" is
> special enough that it really should require positive affirmation /
> consent before doing anything. So I think I'll change the behavior in V2
> to print possible upgrades by default and only take action if a flag is
> specified. I'll have to think a bit more about this...

I think the ability to access what is the current format of various 
repositories will be really useful. This will allow inventory of 
existing repositories to better plan the upgrades.

Looking forward for your V2!

Cheers,

Patch

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -3747,6 +3747,17 @@  def debugtemplate(ui, repo, tmpl, **opts
             displayer.show(repo[r], **props)
         displayer.close()
 
+@command('debugupgraderepo', dryrunopts)
+def debugupgraderepo(ui, repo, **opts):
+    """upgrade a repository to use different features
+
+    During the upgrade, the repository will be locked and no writes will be
+    allowed.
+
+    At times during the upgrade, the repository may not be readable.
+    """
+    raise error.Abort(_('not yet implemented'))
+
 @command('debugwalk', walkopts, _('[OPTION]... [FILE]...'), inferrepo=True)
 def debugwalk(ui, repo, *pats, **opts):
     """show how files match on given patterns"""
diff --git a/tests/test-completion.t b/tests/test-completion.t
--- a/tests/test-completion.t
+++ b/tests/test-completion.t
@@ -109,6 +109,7 @@  Show debug commands if there are no othe
   debugsub
   debugsuccessorssets
   debugtemplate
+  debugupgraderepo
   debugwalk
   debugwireargs
 
@@ -274,6 +275,7 @@  Show all commands + options
   debugsub: rev
   debugsuccessorssets: 
   debugtemplate: rev, define
+  debugupgraderepo: dry-run
   debugwalk: include, exclude
   debugwireargs: three, four, five, ssh, remotecmd, insecure
   files: rev, print0, include, exclude, template, subrepos
diff --git a/tests/test-help.t b/tests/test-help.t
--- a/tests/test-help.t
+++ b/tests/test-help.t
@@ -917,6 +917,8 @@  Test list of internal help commands
                  show set of successors for revision
    debugtemplate
                  parse and apply a template
+   debugupgraderepo
+                 upgrade a repository to use different features
    debugwalk     show how files match on given patterns
    debugwireargs
                  (no help text available)
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
new file mode 100644
--- /dev/null
+++ b/tests/test-upgrade-repo.t
@@ -0,0 +1,5 @@ 
+  $ hg init empty
+  $ cd empty
+  $ hg debugupgraderepo
+  abort: not yet implemented
+  [255]