Patchwork [8,of,8,"] upgrade: support upgrade to/from zstd storage (issue6088)

login
register
mail settings
Submitter Pierre-Yves David
Date March 31, 2019, 3:36 p.m.
Message ID <7acbe373397d9fb6dfb1.1554046584@nodosa.octopoid.net>
Download mbox | patch
Permalink /patch/39425/
State Accepted
Headers show

Comments

Pierre-Yves David - March 31, 2019, 3:36 p.m.
# HG changeset patch
# User Boris Feld <boris.feld@octobus.net>
# Date 1549032662 -3600
#      Fri Feb 01 15:51:02 2019 +0100
# Node ID 7acbe373397d9fb6dfb176b1a69b4ee5f2bf18ba
# Parent  2cfe9983fa92313d58f0420ec62f2341a810343e
# EXP-Topic zstd-revlog
# Available At https://bitbucket.org/octobus/mercurial-devel/
#              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 7acbe373397d
upgrade: support upgrade to/from zstd storage (issue6088)

Now that we have an official config option for a shiny format improvement, we
better make it simple to migrate to/from it.
Gregory Szorc - April 2, 2019, 6:30 p.m.
On Sun, Mar 31, 2019 at 8:39 AM Pierre-Yves David <
pierre-yves.david@ens-lyon.org> wrote:

> # HG changeset patch
> # User Boris Feld <boris.feld@octobus.net>
> # Date 1549032662 -3600
> #      Fri Feb 01 15:51:02 2019 +0100
> # Node ID 7acbe373397d9fb6dfb176b1a69b4ee5f2bf18ba
> # Parent  2cfe9983fa92313d58f0420ec62f2341a810343e
> # EXP-Topic zstd-revlog
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 7acbe373397d
> upgrade: support upgrade to/from zstd storage (issue6088)
>
> Now that we have an official config option for a shiny format improvement,
> we
> better make it simple to migrate to/from it.
>
> diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
> --- a/mercurial/upgrade.py
> +++ b/mercurial/upgrade.py
> @@ -80,6 +80,8 @@ def supporteddestrequirements(repo):
>          'revlogv1',
>          'store',
>          localrepo.SPARSEREVLOG_REQUIREMENT,
> +        'exp-compression-zstd',
> +        'zstd-revlog',
>      }
>
>  def allowednewrequirements(repo):
> @@ -97,6 +99,8 @@ def allowednewrequirements(repo):
>          'fncache',
>          'generaldelta',
>          localrepo.SPARSEREVLOG_REQUIREMENT,
> +        'exp-compression-zstd',
> +        'zstd-revlog',
>      }
>

We shouldn't be adding zstd related requirements to static sets because
zstd support may not be available. We want to make their addition
conditional on zstd support being available.


>
>  def preservedrequirements(repo):
> diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
> --- a/tests/test-upgrade-repo.t
> +++ b/tests/test-upgrade-repo.t
> @@ -854,4 +854,108 @@ Check that we can remove the sparse-revl
>    generaldelta
>    revlogv1
>    store
> +
> +Check upgrading to a zstd revlog
> +--------------------------------
> +
> +upgrade
> +
>

This needs protected behind an `#if zstd`.


> +  $ hg --config format.revlog-compression=zstd debugupgraderepo --run
> >/dev/null
> +  copy of old repository backed up at
> $TESTTMP/sparserevlogrepo/.hg/upgradebackup.* (glob)
> +  the old repository will not be deleted; remove it to free up disk space
> once the upgraded repository is verified
> +  $ hg debugformat -v
> +  format-variant    repo config default
> +  fncache:           yes    yes     yes
> +  dotencode:         yes    yes     yes
> +  generaldelta:      yes    yes     yes
> +  sparserevlog:      yes    yes     yes
> +  plain-cl-delta:    yes    yes     yes
> +  compression:       zstd   zlib    zlib
> +  compression-level: default default default
> +  $ cat .hg/requires
> +  dotencode
> +  fncache
> +  generaldelta
> +  revlogv1
> +  sparserevlog
> +  store
> +  zstd-revlog
> +
> +downgrade
> +
> +  $ hg debugupgraderepo --run
> +  abort: cannot upgrade repository; requirement would be removed:
> zstd-revlog
> +  [255]
> +  $ hg debugformat -v
> +  format-variant    repo config default
> +  fncache:           yes    yes     yes
> +  dotencode:         yes    yes     yes
> +  generaldelta:      yes    yes     yes
> +  sparserevlog:      yes    yes     yes
> +  plain-cl-delta:    yes    yes     yes
> +  compression:       zstd   zlib    zlib
> +  compression-level: default default default
> +  $ cat .hg/requires
> +  dotencode
> +  fncache
> +  generaldelta
> +  revlogv1
> +  sparserevlog
> +  store
> +  zstd-revlog
> +
> +upgrade from hgrc
> +
> +  $ cat >> .hg/hgrc << EOF
> +  > [format]
> +  > revlog-compression=zstd
> +  > EOF
> +  $ hg debugupgraderepo --run
> +  upgrade will perform the following actions:
> +
> +  requirements
> +     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog,
> store, zstd-revlog
> +
> +  beginning upgrade...
> +  repository locked and read-only
> +  creating temporary repository to stage migrated data:
> $TESTTMP/sparserevlogrepo/.hg/upgrade.* (glob)
> +  (it is safe to interrupt this process any time before data migration
> completes)
> +  migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in
> changelog)
> +  migrating 297 bytes in store; 103 bytes tracked data
> +  migrating 1 filelogs containing 1 revisions (64 bytes in store; 0 bytes
> tracked data)
> +  finished migrating 1 filelog revisions across 1 filelogs; change in
> size: 0 bytes
> +  migrating 1 manifests containing 1 revisions (110 bytes in store; 45
> bytes tracked data)
> +  finished migrating 1 manifest revisions across 1 manifests; change in
> size: 0 bytes
> +  migrating changelog containing 1 revisions (123 bytes in store; 58
> bytes tracked data)
> +  finished migrating 1 changelog revisions; change in size: 0 bytes
> +  finished migrating 3 total revisions; total change in store size: 0
> bytes
> +  copying phaseroots
> +  data fully migrated to temporary repository
> +  marking source repository as being upgraded; clients will be unable to
> read from repository
> +  starting in-place swap of repository data
> +  replaced files will be backed up at
> $TESTTMP/sparserevlogrepo/.hg/upgradebackup.* (glob)
> +  replacing store...
> +  store replacement complete; repository was inconsistent for 0.0s
> +  finalizing requirements file and making repository readable again
> +  removing temporary repository $TESTTMP/sparserevlogrepo/.hg/upgrade.*
> (glob)
> +  copy of old repository backed up at
> $TESTTMP/sparserevlogrepo/.hg/upgradebackup.* (glob)
> +  the old repository will not be deleted; remove it to free up disk space
> once the upgraded repository is verified
> +  $ hg debugformat -v
> +  format-variant    repo config default
> +  fncache:           yes    yes     yes
> +  dotencode:         yes    yes     yes
> +  generaldelta:      yes    yes     yes
> +  sparserevlog:      yes    yes     yes
> +  plain-cl-delta:    yes    yes     yes
> +  compression:       zstd   zstd    zlib
> +  compression-level: default default default
> +  $ cat .hg/requires
> +  dotencode
> +  fncache
> +  generaldelta
> +  revlogv1
> +  sparserevlog
> +  store
> +  zstd-revlog
> +
>    $ cd ..
>

Patch

diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -80,6 +80,8 @@  def supporteddestrequirements(repo):
         'revlogv1',
         'store',
         localrepo.SPARSEREVLOG_REQUIREMENT,
+        'exp-compression-zstd',
+        'zstd-revlog',
     }
 
 def allowednewrequirements(repo):
@@ -97,6 +99,8 @@  def allowednewrequirements(repo):
         'fncache',
         'generaldelta',
         localrepo.SPARSEREVLOG_REQUIREMENT,
+        'exp-compression-zstd',
+        'zstd-revlog',
     }
 
 def preservedrequirements(repo):
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
--- a/tests/test-upgrade-repo.t
+++ b/tests/test-upgrade-repo.t
@@ -854,4 +854,108 @@  Check that we can remove the sparse-revl
   generaldelta
   revlogv1
   store
+
+Check upgrading to a zstd revlog
+--------------------------------
+
+upgrade
+
+  $ hg --config format.revlog-compression=zstd debugupgraderepo --run >/dev/null
+  copy of old repository backed up at $TESTTMP/sparserevlogrepo/.hg/upgradebackup.* (glob)
+  the old repository will not be deleted; remove it to free up disk space once the upgraded repository is verified
+  $ hg debugformat -v
+  format-variant    repo config default
+  fncache:           yes    yes     yes
+  dotencode:         yes    yes     yes
+  generaldelta:      yes    yes     yes
+  sparserevlog:      yes    yes     yes
+  plain-cl-delta:    yes    yes     yes
+  compression:       zstd   zlib    zlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlogv1
+  sparserevlog
+  store
+  zstd-revlog
+
+downgrade
+
+  $ hg debugupgraderepo --run
+  abort: cannot upgrade repository; requirement would be removed: zstd-revlog
+  [255]
+  $ hg debugformat -v
+  format-variant    repo config default
+  fncache:           yes    yes     yes
+  dotencode:         yes    yes     yes
+  generaldelta:      yes    yes     yes
+  sparserevlog:      yes    yes     yes
+  plain-cl-delta:    yes    yes     yes
+  compression:       zstd   zlib    zlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlogv1
+  sparserevlog
+  store
+  zstd-revlog
+
+upgrade from hgrc
+
+  $ cat >> .hg/hgrc << EOF
+  > [format]
+  > revlog-compression=zstd
+  > EOF
+  $ hg debugupgraderepo --run
+  upgrade will perform the following actions:
+  
+  requirements
+     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog, store, zstd-revlog
+  
+  beginning upgrade...
+  repository locked and read-only
+  creating temporary repository to stage migrated data: $TESTTMP/sparserevlogrepo/.hg/upgrade.* (glob)
+  (it is safe to interrupt this process any time before data migration completes)
+  migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog)
+  migrating 297 bytes in store; 103 bytes tracked data
+  migrating 1 filelogs containing 1 revisions (64 bytes in store; 0 bytes tracked data)
+  finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes
+  migrating 1 manifests containing 1 revisions (110 bytes in store; 45 bytes tracked data)
+  finished migrating 1 manifest revisions across 1 manifests; change in size: 0 bytes
+  migrating changelog containing 1 revisions (123 bytes in store; 58 bytes tracked data)
+  finished migrating 1 changelog revisions; change in size: 0 bytes
+  finished migrating 3 total revisions; total change in store size: 0 bytes
+  copying phaseroots
+  data fully migrated to temporary repository
+  marking source repository as being upgraded; clients will be unable to read from repository
+  starting in-place swap of repository data
+  replaced files will be backed up at $TESTTMP/sparserevlogrepo/.hg/upgradebackup.* (glob)
+  replacing store...
+  store replacement complete; repository was inconsistent for 0.0s
+  finalizing requirements file and making repository readable again
+  removing temporary repository $TESTTMP/sparserevlogrepo/.hg/upgrade.* (glob)
+  copy of old repository backed up at $TESTTMP/sparserevlogrepo/.hg/upgradebackup.* (glob)
+  the old repository will not be deleted; remove it to free up disk space once the upgraded repository is verified
+  $ hg debugformat -v
+  format-variant    repo config default
+  fncache:           yes    yes     yes
+  dotencode:         yes    yes     yes
+  generaldelta:      yes    yes     yes
+  sparserevlog:      yes    yes     yes
+  plain-cl-delta:    yes    yes     yes
+  compression:       zstd   zstd    zlib
+  compression-level: default default default
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlogv1
+  sparserevlog
+  store
+  zstd-revlog
+
   $ cd ..