| Submitter | via Mercurial-devel |
|---|---|
| Date | May 4, 2017, 5:04 p.m. |
| Message ID | <f4b255ff67d05e329cc3.1493917451@martinvonz.svl.corp.google.com> |
| Download | mbox | patch |
| Permalink | /patch/20431/ |
| State | Changes Requested |
| Headers | show |
Comments
On 05/04/2017 07:04 PM, Martin von Zweigbergk via Mercurial-devel wrote: > # HG changeset patch > # User Martin von Zweigbergk <martinvonz@google.com> > # Date 1458952398 25200 > # Fri Mar 25 17:33:18 2016 -0700 > # Node ID f4b255ff67d05e329cc3d80ce1552f30005f58e9 > # Parent 06058cbb9209df54bfa0aaf9f7d10b836a1e3ee9 > bundle: allow creation of changegroup3 bundles > > With this change, one can use "hg bundle -t v3" to creat changegroup3 > bundles, which support treemanifests and revlog flags. In treemanifest > repos, v3 is the default bundle format. The bundle spec "version" cover a collection of setting for the underlying content. It is not directly correlated to the changegroup version. Tree manifest is still experimental so it seems a bit early to burn a new bundle spec version on this especially since we have other bundle improvement coming (phase, tag cache, obsmarkers, etc). So I don't think we should introduce a new bundlespec version yet. Instead I would recommend the following: 1) introduce a parameter to control de cg-version (cgversion=03?) 2) automatically bump the cg version to '03' if no spec is passed and tree manifest is used, 3) late in the 4.3 cycles consider introducing a v3 with all the new improvements (if some make sense at the user-visible feature level). Cheers,
On Thu, May 4, 2017 at 12:14 PM, Pierre-Yves David < pierre-yves.david@ens-lyon.org> wrote: > > > On 05/04/2017 07:04 PM, Martin von Zweigbergk via Mercurial-devel wrote: > >> # HG changeset patch >> # User Martin von Zweigbergk <martinvonz@google.com> >> # Date 1458952398 25200 >> # Fri Mar 25 17:33:18 2016 -0700 >> # Node ID f4b255ff67d05e329cc3d80ce1552f30005f58e9 >> # Parent 06058cbb9209df54bfa0aaf9f7d10b836a1e3ee9 >> bundle: allow creation of changegroup3 bundles >> >> With this change, one can use "hg bundle -t v3" to creat changegroup3 >> bundles, which support treemanifests and revlog flags. In treemanifest >> repos, v3 is the default bundle format. >> > > The bundle spec "version" cover a collection of setting for the underlying > content. It is not directly correlated to the changegroup version. > > Tree manifest is still experimental so it seems a bit early to burn a new > bundle spec version on this especially since we have other bundle > improvement coming (phase, tag cache, obsmarkers, etc). > > So I don't think we should introduce a new bundlespec version yet. Instead > I would recommend the following: > > 1) introduce a parameter to control de cg-version (cgversion=03?) > 2) automatically bump the cg version to '03' if no spec is passed and tree > manifest is used, > 3) late in the 4.3 cycles consider introducing a v3 with all the new > improvements (if some make sense at the user-visible feature level). > There was confusion about this in IRC. I agree with what Pierre-Yves said: while this will require a "v3" to support cg3, it is still a bit too early in cg3's support cycle to introduce a stable, user-facing bundle spec version. Until we have a better API for controlling bundle generation via bundle specs, I think this should be hacked in via an experimental config option while recycling "v2." Alternatively, we can name the version "expcg3" or something less stable and rename it (presumably to "v3") once the feature is stable. The latter will make it much easier to test the feature and won't result in accidentally producing "v2" bundles that legacy clients choke on. The concern with the "vN" versions is that every Mercurial client with knowledge of that version string has to be capable of handling any future bundle generated with that same "vN" version (or have code for filtering from other data in the clone bundles manifest, preferably from BUNDLESPEC). This means every time there is a new required bundle part, we have to invent a new "vN" or find some other way to alter clonebundles entries so old clients know they can't consume them.
Patch
diff --git a/mercurial/exchange.py b/mercurial/exchange.py --- a/mercurial/exchange.py +++ b/mercurial/exchange.py @@ -40,6 +40,7 @@ # Maps bundle version human names to changegroup versions. _bundlespeccgversions = {'v1': '01', 'v2': '02', + 'v3': '03', 'packed1': 's1', 'bundle2': '02', #legacy } @@ -136,6 +137,9 @@ # Modern compression engines require v2. if compression not in _bundlespecv1compengines: version = 'v2' + # Treemanifests require v3 + if 'treemanifest' in repo.requirements: + version = 'v3' elif spec in _bundlespeccgversions: if spec == 'packed1': compression = 'none' @@ -232,6 +236,8 @@ version = part.params['version'] if version in ('01', '02'): version = 'v2' + elif version == '03': + version = 'v3' else: raise error.Abort(_('changegroup version %s does not have ' 'a known bundlespec') % version, diff --git a/tests/test-bundle-type.t b/tests/test-bundle-type.t --- a/tests/test-bundle-type.t +++ b/tests/test-bundle-type.t @@ -2,6 +2,8 @@ $ cat << EOF >> $HGRCPATH > [format] > usegeneraldelta=yes + > [experimental] + > changegroup3=yes > EOF bundle w/o type option @@ -65,7 +67,8 @@ > cd .. > } - $ for t in "None" "bzip2" "gzip" "none-v2" "v2" "v1" "gzip-v1"; do + $ for t in "None" "bzip2" "gzip" "none-v3" "v3" "none-v2" "v2" "v1" \ + > "gzip-v1"; do > testbundle $t > done % test bundle type None @@ -95,6 +98,24 @@ c35a0f9217e65d1fdb90c936ffa7dbe679f83ddf gzip-v2 + % test bundle type none-v3 + searching for changes + 1 changesets found + HG20\x00\x00 (esc) + Stream params: {} + changegroup -- "sortdict([('version', '03'), ('nbchanges', '1')])" + c35a0f9217e65d1fdb90c936ffa7dbe679f83ddf + none-v3 + + % test bundle type v3 + searching for changes + 1 changesets found + HG20\x00\x00 (esc) + Stream params: sortdict([('Compression', 'BZ')]) + changegroup -- "sortdict([('version', '03'), ('nbchanges', '1')])" + c35a0f9217e65d1fdb90c936ffa7dbe679f83ddf + bzip2-v3 + % test bundle type none-v2 searching for changes 1 changesets found @@ -127,6 +148,9 @@ c35a0f9217e65d1fdb90c936ffa7dbe679f83ddf gzip-v1 + $ hg --config experimental.changegroup3=no -R t1 bundle -t v3 bv3 tv3 + abort: repository does not support bundle version 03 + [255] Compression level can be adjusted for bundle2 bundles diff --git a/tests/test-treemanifest.t b/tests/test-treemanifest.t --- a/tests/test-treemanifest.t +++ b/tests/test-treemanifest.t @@ -867,3 +867,23 @@ 1:678d3574b88c 1:678d3574b88c $ hg --config extensions.strip= strip -r . -q + +Bundle + + $ cd .. + $ hg -R deeprepo bundle --all deeprepo.bundle + 4 changesets found + $ hg debugbundle deeprepo.bundle + Stream params: sortdict([('Compression', 'BZ')]) + changegroup -- "sortdict([('version', '03'), ('nbchanges', '4')])" + 775704be6f529df4abf056a4644c866bd6dd6d3e + 726ec8fc7e58ec32b0049698008851c6040abfa2 + 5a87d5a8da90c45b84639c95eb188b70f5b40a4e + 523e5c631710b290a8d536284a27b9a6eca18bcd + $ hg --config experimental.treemanifest=True init empty-repo2 + $ hg unbundle -R empty-repo2 deeprepo.bundle + adding changesets + adding manifests + adding file changes + added 4 changesets with 18 changes to 8 files + (run 'hg update' to get a working copy)
# HG changeset patch # User Martin von Zweigbergk <martinvonz@google.com> # Date 1458952398 25200 # Fri Mar 25 17:33:18 2016 -0700 # Node ID f4b255ff67d05e329cc3d80ce1552f30005f58e9 # Parent 06058cbb9209df54bfa0aaf9f7d10b836a1e3ee9 bundle: allow creation of changegroup3 bundles With this change, one can use "hg bundle -t v3" to creat changegroup3 bundles, which support treemanifests and revlog flags. In treemanifest repos, v3 is the default bundle format. Note that although changegroup3 is experimental, this patch does not try to hide the v3 format (but also doesn't document it). It's simpler this way and it will still fail unless the user explicitly enables changegroup3 (see test case). Also note that bundlerepo still does not support treemanifests well, so this patch will allow saving changesets to a bundle and unbundling from it, but e.g. "hg log -p repo.bundle" will not work ("hg log repo.bundle" will, though, since it doesn't involve reading manifests). Pulling from it will also not work.