Patchwork [3,of,3,"V4] upgrade: support upgrade to/from zstd storage (issue6088)

login
register
mail settings
Submitter Pierre-Yves David
Date April 16, 2019, 1:26 p.m.
Message ID <821f57c6dda8f3a04997.1555421167@nodosa.octopoid.net>
Download mbox | patch
Permalink /patch/39633/
State Accepted
Headers show

Comments

Pierre-Yves David - April 16, 2019, 1:26 p.m.
# HG changeset patch
# User Pierre-Yves David <pierre-yves.david@octobus.net>
# Date 1549032662 -3600
#      Fri Feb 01 15:51:02 2019 +0100
# Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
# Parent  cbc090e88191176d0d7708e069a98596df78c2c5
# EXP-Topic zstd-revlog
# Available At https://bitbucket.org/octobus/mercurial-devel/
#              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 821f57c6dda8
upgrade: support upgrade to/from zstd storage (issue6088)

Now that we have an official config option for a shiny format improvement, we
better make it simple to migrate to/from it.
Augie Fackler - April 16, 2019, 2:31 p.m.
On Tue, Apr 16, 2019 at 03:26:07PM +0200, Pierre-Yves David wrote:
> # HG changeset patch
> # User Pierre-Yves David <pierre-yves.david@octobus.net>
> # Date 1549032662 -3600
> #      Fri Feb 01 15:51:02 2019 +0100
> # Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
> # Parent  cbc090e88191176d0d7708e069a98596df78c2c5
> # EXP-Topic zstd-revlog
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 821f57c6dda8
> upgrade: support upgrade to/from zstd storage (issue6088)

These look fine, but I'm unclear if we should ship this in the RC or
hold for 5.1. Do you have a preference?
Pierre-Yves David - April 16, 2019, 4:03 p.m.
On 4/16/19 4:31 PM, Augie Fackler wrote:
> On Tue, Apr 16, 2019 at 03:26:07PM +0200, Pierre-Yves David wrote:
>> # HG changeset patch
>> # User Pierre-Yves David <pierre-yves.david@octobus.net>
>> # Date 1549032662 -3600
>> #      Fri Feb 01 15:51:02 2019 +0100
>> # Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
>> # Parent  cbc090e88191176d0d7708e069a98596df78c2c5
>> # EXP-Topic zstd-revlog
>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 821f57c6dda8
>> upgrade: support upgrade to/from zstd storage (issue6088)
> 
> These look fine, but I'm unclear if we should ship this in the RC or
> hold for 5.1. Do you have a preference?

(summary of IRC discussion: Yes, my goal is to get this in 5.0 (disabled 
by default) and I prefer if it gets in).
Gregory Szorc - April 16, 2019, 4:22 p.m.
I would also like to get this in the RC. Makes it more accessible sooner, etc.

> On Apr 16, 2019, at 18:03, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
> 
> 
> 
>> On 4/16/19 4:31 PM, Augie Fackler wrote:
>>> On Tue, Apr 16, 2019 at 03:26:07PM +0200, Pierre-Yves David wrote:
>>> # HG changeset patch
>>> # User Pierre-Yves David <pierre-yves.david@octobus.net>
>>> # Date 1549032662 -3600
>>> #      Fri Feb 01 15:51:02 2019 +0100
>>> # Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
>>> # Parent  cbc090e88191176d0d7708e069a98596df78c2c5
>>> # EXP-Topic zstd-revlog
>>> # Available At https://bitbucket.org/octobus/mercurial-devel/
>>> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 821f57c6dda8
>>> upgrade: support upgrade to/from zstd storage (issue6088)
>> These look fine, but I'm unclear if we should ship this in the RC or
>> hold for 5.1. Do you have a preference?
> 
> (summary of IRC discussion: Yes, my goal is to get this in 5.0 (disabled by default) and I prefer if it gets in).
> 
> 
> -- 
> Pierre-Yves David
Pulkit Goyal - April 16, 2019, 5:09 p.m.
I see that Augie has already queued this series. :)

On Tue, Apr 16, 2019 at 7:53 PM Gregory Szorc <gregory.szorc@gmail.com>
wrote:

> I would also like to get this in the RC. Makes it more accessible sooner,
> etc.
>
> > On Apr 16, 2019, at 18:03, Pierre-Yves David <
> pierre-yves.david@ens-lyon.org> wrote:
> >
> >
> >
> >> On 4/16/19 4:31 PM, Augie Fackler wrote:
> >>> On Tue, Apr 16, 2019 at 03:26:07PM +0200, Pierre-Yves David wrote:
> >>> # HG changeset patch
> >>> # User Pierre-Yves David <pierre-yves.david@octobus.net>
> >>> # Date 1549032662 -3600
> >>> #      Fri Feb 01 15:51:02 2019 +0100
> >>> # Node ID 821f57c6dda8f3a0499717684d617d0c3a83e9e2
> >>> # Parent  cbc090e88191176d0d7708e069a98596df78c2c5
> >>> # EXP-Topic zstd-revlog
> >>> # Available At https://bitbucket.org/octobus/mercurial-devel/
> >>> #              hg pull https://bitbucket.org/octobus/mercurial-devel/
> -r 821f57c6dda8
> >>> upgrade: support upgrade to/from zstd storage (issue6088)
> >> These look fine, but I'm unclear if we should ship this in the RC or
> >> hold for 5.1. Do you have a preference?
> >
> > (summary of IRC discussion: Yes, my goal is to get this in 5.0 (disabled
> by default) and I prefer if it gets in).
> >
> >
> > --
> > Pierre-Yves David
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>

Patch

diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -24,6 +24,10 @@  from . import (
     vfs as vfsmod,
 )
 
+from .utils import (
+    compression,
+)
+
 def requiredsourcerequirements(repo):
     """Obtain requirements required to be present to upgrade a repo.
 
@@ -61,9 +65,16 @@  def supportremovedrequirements(repo):
     the dropped requirement must appear in the returned set for the upgrade
     to be allowed.
     """
-    return {
+    supported = {
         localrepo.SPARSEREVLOG_REQUIREMENT,
     }
+    for name in compression.compengines:
+        engine = compression.compengines[name]
+        if engine.available() and engine.revlogheader():
+            supported.add(b'exp-compression-%s' % name)
+            if engine.name() == 'zstd':
+                supported.add(b'revlog-compression-zstd')
+    return supported
 
 def supporteddestrequirements(repo):
     """Obtain requirements that upgrade supports in the destination.
@@ -73,7 +84,7 @@  def supporteddestrequirements(repo):
 
     Extensions should monkeypatch this to add their custom requirements.
     """
-    return {
+    supported = {
         'dotencode',
         'fncache',
         'generaldelta',
@@ -81,6 +92,13 @@  def supporteddestrequirements(repo):
         'store',
         localrepo.SPARSEREVLOG_REQUIREMENT,
     }
+    for name in compression.compengines:
+        engine = compression.compengines[name]
+        if engine.available() and engine.revlogheader():
+            supported.add(b'exp-compression-%s' % name)
+            if engine.name() == 'zstd':
+                supported.add(b'revlog-compression-zstd')
+    return supported
 
 def allowednewrequirements(repo):
     """Obtain requirements that can be added to a repository during upgrade.
@@ -92,12 +110,19 @@  def allowednewrequirements(repo):
     bad additions because the whitelist approach is safer and will prevent
     future, unknown requirements from accidentally being added.
     """
-    return {
+    supported = {
         'dotencode',
         'fncache',
         'generaldelta',
         localrepo.SPARSEREVLOG_REQUIREMENT,
     }
+    for name in compression.compengines:
+        engine = compression.compengines[name]
+        if engine.available() and engine.revlogheader():
+            supported.add(b'exp-compression-%s' % name)
+            if engine.name() == 'zstd':
+                supported.add(b'revlog-compression-zstd')
+    return supported
 
 def preservedrequirements(repo):
     return set()
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
--- a/tests/test-upgrade-repo.t
+++ b/tests/test-upgrade-repo.t
@@ -854,4 +854,78 @@  Check that we can remove the sparse-revl
   generaldelta
   revlogv1
   store
+
+#if zstd
+
+Check upgrading to a zstd revlog
+--------------------------------
+
+upgrade
+
+  $ hg --config format.revlog-compression=zstd debugupgraderepo --run  --no-backup >/dev/null
+  $ hg debugformat -v
+  format-variant    repo config default
+  fncache:           yes    yes     yes
+  dotencode:         yes    yes     yes
+  generaldelta:      yes    yes     yes
+  sparserevlog:      yes    yes     yes
+  plain-cl-delta:    yes    yes     yes
+  compression:       zstd   zlib    zlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlog-compression-zstd
+  revlogv1
+  sparserevlog
+  store
+
+downgrade
+
+  $ hg debugupgraderepo --run --no-backup > /dev/null
+  $ hg debugformat -v
+  format-variant    repo config default
+  fncache:           yes    yes     yes
+  dotencode:         yes    yes     yes
+  generaldelta:      yes    yes     yes
+  sparserevlog:      yes    yes     yes
+  plain-cl-delta:    yes    yes     yes
+  compression:       zlib   zlib    zlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlogv1
+  sparserevlog
+  store
+
+upgrade from hgrc
+
+  $ cat >> .hg/hgrc << EOF
+  > [format]
+  > revlog-compression=zstd
+  > EOF
+  $ hg debugupgraderepo --run --no-backup > /dev/null
+  $ hg debugformat -v
+  format-variant    repo config default
+  fncache:           yes    yes     yes
+  dotencode:         yes    yes     yes
+  generaldelta:      yes    yes     yes
+  sparserevlog:      yes    yes     yes
+  plain-cl-delta:    yes    yes     yes
+  compression:       zstd   zstd    zlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlog-compression-zstd
+  revlogv1
+  sparserevlog
+  store
+
   $ cd ..
+
+#endif