Patchwork D1952: bundlespec: move computing the bundle contentops in parsebundlespec

login
register
mail settings
Submitter phabricator
Date Jan. 31, 2018, 4:25 p.m.
Message ID <differential-rev-PHID-DREV-h2qxoeuxzm7ygph7hxpq-req@phab.mercurial-scm.org>
Download mbox | patch
Permalink /patch/27094/
State Superseded
Headers show

Comments

phabricator - Jan. 31, 2018, 4:25 p.m.
lothiraldan created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We will introduce a new bundlespec for stream bundle which will influence the
  contentops.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1952

AFFECTED FILES
  mercurial/commands.py
  mercurial/exchange.py

CHANGE DETAILS




To: lothiraldan, #hg-reviewers
Cc: mercurial-devel
phabricator - Feb. 21, 2018, 4:48 a.m.
indygreg requested changes to this revision.
indygreg added a comment.
This revision now requires changes to proceed.


  I like where this is going.
  
  It's worth noting that at some point this will reinvent the //capabilities// mechanism of bundle2. Over the wire protocol today, the client submits its bundle2 capabilities and the server emits parts based on the peer's advertised bundle2 feature support. If you squint, this looks a lot like //content options//. Have you given any consideration to merging the two concepts and having e.g. a bundlespec map to a pre-defined set of bundle2 capabilities?

INLINE COMMENTS

> exchange.py:191
> +    # Set the cg.version
> +    contentops["cg.version"] = version
> +

`version` here does not refer to the changegroup version but rather the bundlespec version.

IMO the changegroup version should be implied by the bundlespec version.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1952

To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
phabricator - March 19, 2018, 8:27 p.m.
indygreg added a comment.


  I took a bit of a deeper look and found a few more things.
  
  Again, I like the direction of this patch. I think our goal for bundle2 generation should be to derive a set of options for its parts up front as quick as possible and pass that data structure into the lower-level bundle2 generation mechanism.

INLINE COMMENTS

> exchange.py:49-51
> +# Maps bundle version with content opts to choose which part to bundle
> +_bundlespeccontentops = {
> +    'v1': {

I *really* like this and this is the direction we need to go. i.e. a bundlespec base version should imply a set of options for generation of the bundle.

> exchange.py:89
>  
>      Returns a 3-tuple of (compression, version, parameters). Compression will
>      be ``None`` if not in strict mode and a compression isn't defined.

This comment needs updated.

> exchange.py:193
> +
> +    return compression, version, params, contentops
>  

At the point we're returning yet another element, it might be worth converting the return into an `attr`-derived class. We're starting to use those a bit to define data structures that are too large for tuples but don't yet merit a normal class.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1952

To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
phabricator - March 26, 2018, 8:24 p.m.
lothiraldan added a comment.


  In https://phab.mercurial-scm.org/D1952#38770, @indygreg wrote:
  
  > I like where this is going.
  >
  > It's worth noting that at some point this will reinvent the //capabilities// mechanism of bundle2. Over the wire protocol today, the client submits its bundle2 capabilities and the server emits parts based on the peer's advertised bundle2 feature support. If you squint, this looks a lot like //content options//. Have you given any consideration to merging the two concepts and having e.g. a bundlespec map to a pre-defined set of bundle2 capabilities?
  
  
  Both concepts seem intimately intricated but I was afraid of the complexity of the series that actually merge them and all the potential backward-incompatible implications. Do you have a specific design in mind we could iterate on?
  
  I will update this series according to your feedbacks.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1952

To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
phabricator - March 30, 2018, 5:04 p.m.
indygreg accepted this revision.
indygreg added a comment.
This revision is now accepted and ready to land.


  Thanks for following up. This looks great!

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1952

To: lothiraldan, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel

Patch

diff --git a/mercurial/exchange.py b/mercurial/exchange.py
--- a/mercurial/exchange.py
+++ b/mercurial/exchange.py
@@ -46,6 +46,20 @@ 
                          'bundle2': '02', #legacy
                         }
 
+# Maps bundle version with content opts to choose which part to bundle
+_bundlespeccontentops = {
+    'v1': {
+        'changegroup': True,
+        'obsolescence': False,
+        'phases': False
+    },
+    'v2': {
+        'changegroup': True,
+        'obsolescence': False,
+        'phases': False
+    }
+}
+
 # Compression engines allowed in version 1. THIS SHOULD NEVER CHANGE.
 _bundlespecv1compengines = {'gzip', 'bzip2', 'none'}
 
@@ -165,11 +179,18 @@ 
                     _('missing support for repository features: %s') %
                       ', '.join(sorted(missingreqs)))
 
+    # Compute contentops based on the version
+    contentops = _bundlespeccontentops.get(version, {}).copy()
+
     if not externalnames:
         engine = util.compengines.forbundlename(compression)
         compression = engine.bundletype()[1]
         version = _bundlespeccgversions[version]
-    return compression, version, params
+
+    # Set the cg.version
+    contentops["cg.version"] = version
+
+    return compression, version, params, contentops
 
 def readbundle(ui, fh, fname, vfs=None):
     header = changegroup.readexactly(fh, 4)
@@ -2107,7 +2128,7 @@ 
             # component of the BUNDLESPEC.
             if key == 'BUNDLESPEC':
                 try:
-                    comp, version, params = parsebundlespec(repo, value,
+                    comp, version, params, _ = parsebundlespec(repo, value,
                                                             externalnames=True)
                     attrs['COMPRESSION'] = comp
                     attrs['VERSION'] = version
@@ -2135,7 +2156,7 @@ 
         spec = entry.get('BUNDLESPEC')
         if spec:
             try:
-                comp, version, params = parsebundlespec(repo, spec, strict=True)
+                comp, version, params, _ = parsebundlespec(repo, spec, strict=True)
 
                 # If a stream clone was requested, filter out non-streamclone
                 # entries.
diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -1189,7 +1189,7 @@ 
 
     bundletype = opts.get('type', 'bzip2').lower()
     try:
-        bcompression, cgversion, params = exchange.parsebundlespec(
+        bcompression, cgversion, params, contentopts = exchange.parsebundlespec(
                 repo, bundletype, strict=False)
     except error.UnsupportedBundleSpecification as e:
         raise error.Abort(str(e),
@@ -1256,12 +1256,13 @@ 
     if complevel is not None:
         compopts['level'] = complevel
 
-
-    contentopts = {'cg.version': cgversion, 'changegroup': True}
+    # Allow overriding the bundling of obsmarker in phases through
+    # configuration while we don't have a bundle version that include them
     if repo.ui.configbool('experimental', 'evolution.bundle-obsmarker'):
         contentopts['obsolescence'] = True
     if repo.ui.configbool('experimental', 'bundle-phases'):
         contentopts['phases'] = True
+
     bundle2.writenewbundle(ui, repo, 'bundle', fname, bversion, outgoing,
                            contentopts, compression=bcompression,
                            compopts=compopts)