Patchwork [3,of,3,RFC,cg3] changegroup: introduce cg3, which has support for exchanging treemanifests

login
register
mail settings
Submitter Augie Fackler
Date Dec. 3, 2015, 3:58 p.m.
Message ID <1240273d0971ff1d5e59.1449158317@arthedain.pit.corp.google.com>
Download mbox | patch
Permalink /patch/11789/
State Superseded
Delegated to: Martin von Zweigbergk
Headers show

Comments

Augie Fackler - Dec. 3, 2015, 3:58 p.m.
# HG changeset patch
# User Augie Fackler <augie@google.com>
# Date 1449092255 21600
#      Wed Dec 02 15:37:35 2015 -0600
# Node ID 1240273d0971ff1d5e596ffea69cf34a07335d92
# Parent  79ad570337631038a08b75957b2890975221e089
# EXP-Topic cg3
changegroup: introduce cg3, which has support for exchanging treemanifests

I'm not entirely happy with using a trailing / on a "file" entry for
transferring a treemanifest. We've discussed putting some flags on
each file header[0], but I'm unconvinced that's actually any better:
if we were going to add another feature to the cg format we'd still be
doing a version bump anyway to cg4, so I'm inclined to not spend time
coming up with a more sophisticated format until we actually know what
the next feature we want to stuff in a changegroup will be.

Test changes outside test-treemanifest.t are only due to the new CG3
bundlecap showing up in the wire protocol.

Many thanks to adgar@google.com for helping me with various odd
corners of the API.

0: It's not hard refactoring, nor is it a lot of work. I'm just
disinclined to do speculative work when it's not clear what the
customer would actually be.
Augie Fackler - Dec. 3, 2015, 5:03 p.m.
On Thu, Dec 3, 2015 at 10:58 AM, Augie Fackler <raf@durin42.com> wrote:
> @@ -676,10 +699,26 @@ class cg1packer(object):
>                          fclnode = fclnodes.setdefault(n, clnode)
>                          if clrevorder[clnode] < clrevorder[fclnode]:
>                              fclnodes[n] = clnode
> +                        # gather list of changed treemanifest nodes
> +                        if '/' in f and ('treemanifest' in
> +                                         self._repo.requirements):
> +                            dirs = f.split('/')[:-1]
> +                            submf = mdata
> +                            path = []
> +                            for sd in dirs:
> +                                submf = submf._dirs[sd + '/']
> +                                fullpath = posixpath.join(*(path + [sd]))
> +                                path.append(sd)
> +                                tmfclnodes = tmfnodes.setdefault(
> +                                    fullpath, {})
> +                                tmfclnodes.setdefault(submf.node(), clnode)
> +                                if clrevorder[clnode] < clrevorder[fclnode]:
> +                                    tmfclnodes[n] = clnode
>              return clnode
>
>          mfnodes = self.prune(ml, mfs, commonrevs)
> -        for x in self._packmanifests(mfnodes, lookupmflinknode):
> +        for x in self._packmanifests(
> +            mfnodes, tmfnodes, lookupmflinknode):
>              yield x
>
>          mfs.clear()
> @@ -797,9 +836,32 @@ class cg2packer(cg1packer):
>      def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
>          return struct.pack(self.deltaheader, node, p1n, p2n, basenode, linknode)
>
> +class cg3packer(cg2packer):
> +    version = '03'
> +
> +    def _packmanifests(self, mfnodes, tmfnodes, lookuplinknode):
> +        # Note that debug prints are super confusing in this code, as
> +        # tmfnodes gets populated by the calls to lookuplinknode in
> +        # the superclass's manifest packer. In the future we should
> +        # probably see if we can refactor this somehow to be less
> +        # confusing.
> +        for x in super(cg3packer, self)._packmanifests(
> +            mfnodes, {}, lookuplinknode):
> +            yield x
> +        dirlog = self._repo.manifest.dirlog
> +        for name, nodes in tmfnodes.iteritems():
> +            # For now, directory headers are simply file headers with
> +            # a trailing '/' on the path.
> +            yield self.fileheader(name + '/')
> +            for chunk in self.group(nodes, dirlog(name), nodes.get):
> +                yield chunk
> +
> +


The part I've quoted in this message is the part that I would like
comments from adgar on, with an eye towards how we make this work with
narrowhg.
Martin von Zweigbergk - Dec. 3, 2015, 5:42 p.m.
On Thu, Dec 3, 2015 at 7:58 AM Augie Fackler <raf@durin42.com> wrote:

> # HG changeset patch
> # User Augie Fackler <augie@google.com>
> # Date 1449092255 21600
> #      Wed Dec 02 15:37:35 2015 -0600
> # Node ID 1240273d0971ff1d5e596ffea69cf34a07335d92
> # Parent  79ad570337631038a08b75957b2890975221e089
> # EXP-Topic cg3
> changegroup: introduce cg3, which has support for exchanging treemanifests
>
> I'm not entirely happy with using a trailing / on a "file" entry for
> transferring a treemanifest. We've discussed putting some flags on
> each file header[0], but I'm unconvinced that's actually any better:
> if we were going to add another feature to the cg format we'd still be
> doing a version bump anyway to cg4, so I'm inclined to not spend time
> coming up with a more sophisticated format until we actually know what
> the next feature we want to stuff in a changegroup will be.
>
> Test changes outside test-treemanifest.t are only due to the new CG3
> bundlecap showing up in the wire protocol.
>
> Many thanks to adgar@google.com for helping me with various odd
> corners of the API.
>
> 0: It's not hard refactoring, nor is it a lot of work. I'm just
> disinclined to do speculative work when it's not clear what the
> customer would actually be.
>
> diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
> --- a/mercurial/changegroup.py
> +++ b/mercurial/changegroup.py
> @@ -8,6 +8,7 @@
>  from __future__ import absolute_import
>
>  import os
> +import posixpath
>  import struct
>  import tempfile
>  import weakref
> @@ -497,6 +498,14 @@ class cg2unpacker(cg1unpacker):
>          node, p1, p2, deltabase, cs = headertuple
>          return node, p1, p2, deltabase, cs
>
> +class cg3unpacker(cg2unpacker):
> +    """Unpacker for cg3 streams.
> +
> +    cg3 streams add support for exchanging treemanifests, so the only
> +    thing that changes is the version number.
> +    """
> +    version = '03'
> +
>  class headerlessfixup(object):
>      def __init__(self, fh, h):
>          self._h = h
> @@ -594,7 +603,7 @@ class cg1packer(object):
>          rr, rl = revlog.rev, revlog.linkrev
>          return [n for n in missing if rl(rr(n)) not in commonrevs]
>
> -    def _packmanifests(self, mfnodes, lookuplinknode):
> +    def _packmanifests(self, mfnodes, tmfnodes, lookuplinknode):
>          """Pack flat manifests into a changegroup stream."""
>          ml = self._repo.manifest
>          size = 0
> @@ -603,6 +612,11 @@ class cg1packer(object):
>              size += len(chunk)
>              yield chunk
>          self._verbosenote(_('%8.i (manifests)\n') % size)
> +        # It looks odd to assert this here, but tmfnodes doesn't get
> +        # filled in until after we've called lookuplinknode for
> +        # sending root manifests, so the only way to tell the streams
> +        # got crossed is to check after we've done all the work.
> +        assert not tmfnodes
>

I think it would be clearer to override the entire generate() method and
add call the cg2packer.generate() from there if not using tree manifests
and then dedicate the rest of the method to tree manifests. It would mean
more code duplication, but some of that could be extracted to helpers. I
tried it and I think it removes a lot of confusion.


>
>      def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
>          '''yield a sequence of changegroup chunks (strings)'''
> @@ -612,6 +626,7 @@ class cg1packer(object):
>
>          clrevorder = {}
>          mfs = {} # needed manifests
> +        tmfnodes = {}
>          fnodes = {} # needed file nodes
>          changedfiles = set()
>
> @@ -649,6 +664,11 @@ class cg1packer(object):
>          # simply take the slowpath, which already has the 'clrevorder'
> logic.
>          # This was also fixed in cc0ff93d0c0c.
>          fastpathlinkrev = fastpathlinkrev and not self._reorder
> +        # Treemanifests don't work correctly with fastpathlinkrev
> +        # either, because we don't discover which directory nodes to
> +        # send along with files. This could probably be fixed.
> +        fastpathlinkrev = fastpathlinkrev and (
> +            'treemanifest' not in self._repo.requirements)
>          # Callback for the manifest, used to collect linkrevs for filelog
>          # revisions.
>          # Returns the linkrev node (collected in lookupcl).
> @@ -659,11 +679,14 @@ class cg1packer(object):
>
>              SIDE EFFECT:
>
> -              fclnodes gets populated with the list of relevant
> -              file nodes if we're not using fastpathlinkrev.
> +              1) fclnodes gets populated with the list of relevant
> +                 file nodes if we're not using fastpathlinkrev
> +              2) When treemanifests are in use, collects treemanifest
> nodes
> +                 to send
>
> -            Note that this means you can't trust fclnodes until
> -            after manifests have been sent to the client.
> +            Note that this means manifests must be completely sent to
> +            the client before you can trust the list of files and
> +            treemanifests to send.
>              """
>              clnode = mfs[x]
>              if not fastpathlinkrev:
> @@ -676,10 +699,26 @@ class cg1packer(object):
>                          fclnode = fclnodes.setdefault(n, clnode)
>                          if clrevorder[clnode] < clrevorder[fclnode]:
>                              fclnodes[n] = clnode
> +                        # gather list of changed treemanifest nodes
> +                        if '/' in f and ('treemanifest' in
> +                                         self._repo.requirements):
> +                            dirs = f.split('/')[:-1]
> +                            submf = mdata
> +                            path = []
> +                            for sd in dirs:
> +                                submf = submf._dirs[sd + '/']
>

Using the "private" members in treemanifest directly is a little scary. The
class takes care to load the manifest in all public methods, but accessing
_dirs directly does not do that. I think it works here because the
readfast() method called into readdelta(), which called into
_slowreaddelta(), which populates the manifest with a diff. However, if the
delta parent is not a parent, readfast() would not call readdelta(), it
instead calls read(), which returns an uninitialized (lazily loaded)
manifest. Besides the problem of it not being loaded (I *think*; we'd have
to verify), it would be bad for narrowhg, which does not want the entire
manifest iterated over.

I think cg3packer.generate() should *always* call readdelta() for
treemanifests (until we implement the fast path).


> +                                fullpath = posixpath.join(*(path + [sd]))
>

I think fullpath can be replaced by submf.dir()


> +                                path.append(sd)
> +                                tmfclnodes = tmfnodes.setdefault(
> +                                    fullpath, {})
> +                                tmfclnodes.setdefault(submf.node(),
> clnode)
> +                                if clrevorder[clnode] <
> clrevorder[fclnode]:
> +                                    tmfclnodes[n] = clnode
>              return clnode
>
>          mfnodes = self.prune(ml, mfs, commonrevs)
> -        for x in self._packmanifests(mfnodes, lookupmflinknode):
> +        for x in self._packmanifests(
> +            mfnodes, tmfnodes, lookupmflinknode):
>              yield x
>
>          mfs.clear()
> @@ -797,9 +836,32 @@ class cg2packer(cg1packer):
>      def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
>          return struct.pack(self.deltaheader, node, p1n, p2n, basenode,
> linknode)
>
> +class cg3packer(cg2packer):
> +    version = '03'
> +
> +    def _packmanifests(self, mfnodes, tmfnodes, lookuplinknode):
> +        # Note that debug prints are super confusing in this code, as
> +        # tmfnodes gets populated by the calls to lookuplinknode in
> +        # the superclass's manifest packer. In the future we should
> +        # probably see if we can refactor this somehow to be less
> +        # confusing.
> +        for x in super(cg3packer, self)._packmanifests(
> +            mfnodes, {}, lookuplinknode):
> +            yield x
> +        dirlog = self._repo.manifest.dirlog
> +        for name, nodes in tmfnodes.iteritems():
> +            # For now, directory headers are simply file headers with
> +            # a trailing '/' on the path.
> +            yield self.fileheader(name + '/')
> +            for chunk in self.group(nodes, dirlog(name), nodes.get):
> +                yield chunk
> +
> +
>  packermap = {'01': (cg1packer, cg1unpacker),
>               # cg2 adds support for exchanging generaldelta
>               '02': (cg2packer, cg2unpacker),
> +             # cg3 adds support for exchanging treemanifests
> +             '03': (cg3packer, cg3unpacker),
>  }
>
>  def _changegroupinfo(repo, nodes, source):
> @@ -926,15 +988,33 @@ def _addchangegroupfiles(repo, source, r
>          f = chunkdata["filename"]
>          repo.ui.debug("adding %s revisions\n" % f)
>          pr()
> -        fl = repo.file(f)
> +        directory = (f[-1] == '/')
> +        if directory:
> +            # a directory using treemanifests
> +            # TODO fixup repo requirements safely
> +            if 'treemanifest' not in repo.requirements:
> +                if not wasempty:
> +                    raise error.Abort(_(
> +                        "bundle contains tree manifests, but local repo
> is "
> +                        "non-empty and does not use tree manifests"))
> +                repo.requirements.add('treemanifest')
> +                repo._applyopenerreqs()
> +                repo._writerequirements()
> +                repo.manifest._treeondisk = True
> +                repo.manifest._treeinmem = True
>

This would not be necessary if the requirement had been added to the repo
before the manifest revlog was accessed. How do we handle this elsewhere?
Is this check what we would want the client to run into:
https://selenic.com/hg/file/7e1fac6c0a9c/mercurial/exchange.py#l1103


> +            fl = repo.manifest.dirlog(f)
> +        else:
> +            fl = repo.file(f)
>          o = len(fl)
>          try:
>              if not fl.addgroup(source, revmap, trp):
>                  raise error.Abort(_("received file revlog group is
> empty"))
>          except error.CensoredBaseError as e:
>              raise error.Abort(_("received delta base is censored: %s") %
> e)
> -        revisions += len(fl) - o
> -        files += 1
> +        if not directory:
> +            revisions += len(fl) - o
> +            files += 1
> +        # FIXME what is needfiles
>          if f in needfiles:
>              needs = needfiles[f]
>              for new in xrange(o, len(fl)):
> diff --git a/tests/test-acl.t b/tests/test-acl.t
> --- a/tests/test-acl.t
> +++ b/tests/test-acl.t
> @@ -99,13 +99,13 @@ Extension disabled for lack of a hook
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -162,13 +162,13 @@ Extension disabled for lack of acl.sourc
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -228,13 +228,13 @@ No [acl.allow]/[acl.deny]
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -304,13 +304,13 @@ Empty [acl.allow]
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -369,13 +369,13 @@ fred is allowed inside foo/
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -439,13 +439,13 @@ Empty [acl.deny]
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -506,13 +506,13 @@ fred is allowed inside foo/, but not foo
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -578,13 +578,13 @@ fred is allowed inside foo/, but not foo
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -647,13 +647,13 @@ fred is allowed inside foo/, but not foo
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -718,13 +718,13 @@ barney is allowed everywhere
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -801,13 +801,13 @@ wilma can change files with a .txt exten
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -879,13 +879,13 @@ file specified by acl.config does not ex
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -952,13 +952,13 @@ betty is allowed inside foo/ by a acl.co
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1036,13 +1036,13 @@ acl.config can set only [acl.allow]/[acl
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1122,13 +1122,13 @@ fred is always allowed
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1204,13 +1204,13 @@ no one is allowed inside foo/Bar/
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1280,13 +1280,13 @@ OS-level groups
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1363,13 +1363,13 @@ OS-level groups
>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>    911600dab2ae7a9baff75958b84fe606851ce955
>    bundle2-output-bundle: "HG20", 4 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1478,14 +1478,14 @@ No branch acls specified
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1566,14 +1566,14 @@ Branch acl deny test
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1640,14 +1640,14 @@ Branch acl empty allow test
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1710,14 +1710,14 @@ Branch acl allow other
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1774,14 +1774,14 @@ Branch acl allow other
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1867,14 +1867,14 @@ push foobar into the remote
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -1959,14 +1959,14 @@ Branch acl conflicting deny
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -2028,14 +2028,14 @@ User 'astro' must not be denied
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> @@ -2114,14 +2114,14 @@ Non-astro users must be denied
>    911600dab2ae7a9baff75958b84fe606851ce955
>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>    bundle2-output-bundle: "HG20", 5 parts total
> -  bundle2-output-part: "replycaps" 155 bytes payload
> +  bundle2-output-part: "replycaps" 158 bytes payload
>    bundle2-output-part: "check:heads" streamed payload
>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
> payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>    bundle2-input-bundle: with-transaction
>    bundle2-input-part: "replycaps" supported
> -  bundle2-input-part: total payload size 155
> +  bundle2-input-part: total payload size 158
>    bundle2-input-part: "check:heads" supported
>    bundle2-input-part: total payload size 20
>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
> diff --git a/tests/test-clonebundles.t b/tests/test-clonebundles.t
> --- a/tests/test-clonebundles.t
> +++ b/tests/test-clonebundles.t
> @@ -33,7 +33,7 @@ Feature disabled by default
>    $ cat server/access.log
>    * - - [*] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
>    * - - [*] "GET /?cmd=batch HTTP/1.1" 200 -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
> -  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks
> (glob)
> +  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks
> (glob)
>    * - - [*] "GET /?cmd=listkeys HTTP/1.1" 200 -
> x-hgarg-1:namespace=phases (glob)
>
>    $ cat >> $HGRCPATH << EOF
> @@ -53,7 +53,7 @@ Missing manifest should not result in se
>    $ tail -4 server/access.log
>    * - - [*] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
>    * - - [*] "GET /?cmd=batch HTTP/1.1" 200 -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
> -  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks
> (glob)
> +  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks
> (glob)
>    * - - [*] "GET /?cmd=listkeys HTTP/1.1" 200 -
> x-hgarg-1:namespace=phases (glob)
>
>  Empty manifest file results in retrieval
> diff --git a/tests/test-hgweb-commands.t b/tests/test-hgweb-commands.t
> --- a/tests/test-hgweb-commands.t
> +++ b/tests/test-hgweb-commands.t
> @@ -1897,7 +1897,7 @@ capabilities
>    $ get-with-headers.py 127.0.0.1:$HGPORT '?cmd=capabilities'; echo
>    200 Script output follows
>
> -  lookup changegroupsubset branchmap pushkey known getbundle unbundlehash
> batch
> bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1*%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 (glob)
> +  lookup changegroupsubset branchmap pushkey known getbundle unbundlehash
> batch
> bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
>
>  heads
>
> @@ -2128,7 +2128,7 @@ capabilities
>    batch
>    stream-preferred
>    streamreqs=generaldelta,revlogv1
> -
> bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
> +
> bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>    unbundle=HG10GZ,HG10BZ,HG10UN
>    httpheader=1024
>
> diff --git a/tests/test-http-proxy.t b/tests/test-http-proxy.t
> --- a/tests/test-http-proxy.t
> +++ b/tests/test-http-proxy.t
> @@ -110,21 +110,21 @@ do not use the proxy if it is in the no
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=branchmap HTTP/1.1" - -
> (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=stream_out HTTP/1.1" - -
> (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D83180e7845de420a1bb46896fd5fe05294f8d629
> (glob)
> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=83180e7845de420a1bb46896fd5fe05294f8d629&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=83180e7845de420a1bb46896fd5fe05294f8d629&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
> x-hgarg-1:namespace=phases (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" -
> - (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
> x-hgarg-1:namespace=phases (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" -
> - (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
> x-hgarg-1:namespace=phases (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" -
> - (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
> x-hgarg-1:namespace=phases (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" -
> - (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
> (glob)
>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
> x-hgarg-1:namespace=phases (glob)
> diff --git a/tests/test-http.t b/tests/test-http.t
> --- a/tests/test-http.t
> +++ b/tests/test-http.t
> @@ -262,12 +262,12 @@ test http authentication
>    "GET /?cmd=stream_out HTTP/1.1" 401 -
>    "GET /?cmd=stream_out HTTP/1.1" 200 -
>    "GET /?cmd=batch HTTP/1.1" 200 -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D5fed3813f7f5e1824344fdc9cf8f63bb662c292d
> -  "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
> +  "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
>    "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
>    "GET /?cmd=capabilities HTTP/1.1" 200 -
>    "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D
> -  "GET /?cmd=getbundle HTTP/1.1" 401 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
> -  "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
> +  "GET /?cmd=getbundle HTTP/1.1" 401 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
> +  "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
>    "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
>    "GET /?cmd=capabilities HTTP/1.1" 200 -
>    "GET /?cmd=lookup HTTP/1.1" 200 - x-hgarg-1:key=tip
> diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t
> --- a/tests/test-setdiscovery.t
> +++ b/tests/test-setdiscovery.t
> @@ -350,7 +350,7 @@ Test actual protocol when pulling one ne
>    $ cut -d' ' -f6- access.log | grep -v cmd=known # cmd=known uses random
> sampling
>    "GET /?cmd=capabilities HTTP/1.1" 200 -
>    "GET /?cmd=batch HTTP/1.1" 200 -
> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D513314ca8b3ae4dac8eec56966265b00fcf866db
> -  "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=513314ca8b3ae4dac8eec56966265b00fcf866db&heads=e64a39e7da8b0d54bc63e81169aff001c13b3477
> +  "GET /?cmd=getbundle HTTP/1.1" 200 -
> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=513314ca8b3ae4dac8eec56966265b00fcf866db&heads=e64a39e7da8b0d54bc63e81169aff001c13b3477
>    "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
>    $ cat errors.log
>
> diff --git a/tests/test-ssh-bundle1.t b/tests/test-ssh-bundle1.t
> --- a/tests/test-ssh-bundle1.t
> +++ b/tests/test-ssh-bundle1.t
> @@ -462,8 +462,8 @@ debug output
>    running python ".*/dummyssh" user@dummy ('|")hg -R remote serve
> --stdio('|") (re)
>    sending hello command
>    sending between command
> -  remote: 371
> -  remote: capabilities: lookup changegroupsubset branchmap pushkey known
> getbundle unbundlehash batch streamreqs=generaldelta,revlogv1
> bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
> +  remote: 376
> +  remote: capabilities: lookup changegroupsubset branchmap pushkey known
> getbundle unbundlehash batch streamreqs=generaldelta,revlogv1
> bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
>    remote: 1
>    preparing listkeys for "bookmarks"
>    sending listkeys command
> diff --git a/tests/test-ssh.t b/tests/test-ssh.t
> --- a/tests/test-ssh.t
> +++ b/tests/test-ssh.t
> @@ -454,8 +454,8 @@ debug output
>    running python ".*/dummyssh" user@dummy ('|")hg -R remote serve
> --stdio('|") (re)
>    sending hello command
>    sending between command
> -  remote: 371
> -  remote: capabilities: lookup changegroupsubset branchmap pushkey known
> getbundle unbundlehash batch streamreqs=generaldelta,revlogv1
> bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
> +  remote: 376
> +  remote: capabilities: lookup changegroupsubset branchmap pushkey known
> getbundle unbundlehash batch streamreqs=generaldelta,revlogv1
> bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
>    remote: 1
>    query 1; heads
>    sending batch command
> diff --git a/tests/test-treemanifest.t b/tests/test-treemanifest.t
> --- a/tests/test-treemanifest.t
> +++ b/tests/test-treemanifest.t
> @@ -388,3 +388,50 @@ within that.
>    $ mv oldmf2 .hg/store/meta/b/foo
>    $ mv oldmf3 .hg/store/meta/b/bar/orange
>
> +Test cloning a treemanifest repo over http.
> +  $ hg serve -p $HGPORT -d --pid-file=hg.pid --errorlog=errors.log
> +  $ cat hg.pid >> $DAEMON_PIDS
> +  $ cd ..
> +We can clone even with the knob turned off and we'll get a treemanifest
> repo.
> +  $ hg clone --config experimental.treemanifest=False \
> +  >   http://localhost:$HGPORT deepclone
> +  requesting all changes
> +  adding changesets
> +  adding manifests
> +  adding file changes
> +  added 1 changesets with 8 changes to 8 files
> +  updating to branch default
> +  8 files updated, 0 files merged, 0 files removed, 0 files unresolved
> +No server errors.
> +  $ cat deeprepo/errors.log
> +requires got updated to include treemanifest
> +  $ cat deepclone/.hg/requires | grep treemanifest
> +  treemanifest
> +Tree manifest revlogs exist.
> +  $ find deepclone/.hg/store/meta | sort
> +  deepclone/.hg/store/meta
> +  deepclone/.hg/store/meta/a
> +  deepclone/.hg/store/meta/a/00manifest.i
> +  deepclone/.hg/store/meta/b
> +  deepclone/.hg/store/meta/b/00manifest.i
> +  deepclone/.hg/store/meta/b/bar
> +  deepclone/.hg/store/meta/b/bar/00manifest.i
> +  deepclone/.hg/store/meta/b/bar/orange
> +  deepclone/.hg/store/meta/b/bar/orange/00manifest.i
> +  deepclone/.hg/store/meta/b/bar/orange/fly
> +  deepclone/.hg/store/meta/b/bar/orange/fly/00manifest.i
> +  deepclone/.hg/store/meta/b/foo
> +  deepclone/.hg/store/meta/b/foo/00manifest.i
> +  deepclone/.hg/store/meta/b/foo/apple
> +  deepclone/.hg/store/meta/b/foo/apple/00manifest.i
> +  deepclone/.hg/store/meta/b/foo/apple/bees
> +  deepclone/.hg/store/meta/b/foo/apple/bees/00manifest.i
> +Verify passes.
> +  $ cd deepclone
> +  $ hg verify
> +  checking changesets
> +  checking manifests
> +  crosschecking files in changesets and manifests
> +  checking files
> +  8 files, 1 changesets, 8 total revisions
> +  $ cd ..
>
Martin von Zweigbergk - Dec. 3, 2015, 6:57 p.m.
Gmail tells me I've sent the below, but the recipients seem to not have
gotten it. Let's see if replying works...

On Thu, Dec 3, 2015, 09:42 Martin von Zweigbergk <martinvonz@google.com>
wrote:

> On Thu, Dec 3, 2015 at 7:58 AM Augie Fackler <raf@durin42.com> wrote:
>
>> # HG changeset patch
>> # User Augie Fackler <augie@google.com>
>> # Date 1449092255 21600
>> #      Wed Dec 02 15:37:35 2015 -0600
>> # Node ID 1240273d0971ff1d5e596ffea69cf34a07335d92
>> # Parent  79ad570337631038a08b75957b2890975221e089
>> # EXP-Topic cg3
>> changegroup: introduce cg3, which has support for exchanging treemanifests
>>
>> I'm not entirely happy with using a trailing / on a "file" entry for
>> transferring a treemanifest. We've discussed putting some flags on
>> each file header[0], but I'm unconvinced that's actually any better:
>> if we were going to add another feature to the cg format we'd still be
>> doing a version bump anyway to cg4, so I'm inclined to not spend time
>> coming up with a more sophisticated format until we actually know what
>> the next feature we want to stuff in a changegroup will be.
>>
>> Test changes outside test-treemanifest.t are only due to the new CG3
>> bundlecap showing up in the wire protocol.
>>
>> Many thanks to adgar@google.com for helping me with various odd
>> corners of the API.
>>
>> 0: It's not hard refactoring, nor is it a lot of work. I'm just
>> disinclined to do speculative work when it's not clear what the
>> customer would actually be.
>>
>> diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
>> --- a/mercurial/changegroup.py
>> +++ b/mercurial/changegroup.py
>> @@ -8,6 +8,7 @@
>>  from __future__ import absolute_import
>>
>>  import os
>> +import posixpath
>>  import struct
>>  import tempfile
>>  import weakref
>> @@ -497,6 +498,14 @@ class cg2unpacker(cg1unpacker):
>>          node, p1, p2, deltabase, cs = headertuple
>>          return node, p1, p2, deltabase, cs
>>
>> +class cg3unpacker(cg2unpacker):
>> +    """Unpacker for cg3 streams.
>> +
>> +    cg3 streams add support for exchanging treemanifests, so the only
>> +    thing that changes is the version number.
>> +    """
>> +    version = '03'
>> +
>>  class headerlessfixup(object):
>>      def __init__(self, fh, h):
>>          self._h = h
>> @@ -594,7 +603,7 @@ class cg1packer(object):
>>          rr, rl = revlog.rev, revlog.linkrev
>>          return [n for n in missing if rl(rr(n)) not in commonrevs]
>>
>> -    def _packmanifests(self, mfnodes, lookuplinknode):
>> +    def _packmanifests(self, mfnodes, tmfnodes, lookuplinknode):
>>          """Pack flat manifests into a changegroup stream."""
>>          ml = self._repo.manifest
>>          size = 0
>> @@ -603,6 +612,11 @@ class cg1packer(object):
>>              size += len(chunk)
>>              yield chunk
>>          self._verbosenote(_('%8.i (manifests)\n') % size)
>> +        # It looks odd to assert this here, but tmfnodes doesn't get
>> +        # filled in until after we've called lookuplinknode for
>> +        # sending root manifests, so the only way to tell the streams
>> +        # got crossed is to check after we've done all the work.
>> +        assert not tmfnodes
>>
>
> I think it would be clearer to override the entire generate() method and
> add call the cg2packer.generate() from there if not using tree manifests
> and then dedicate the rest of the method to tree manifests. It would mean
> more code duplication, but some of that could be extracted to helpers. I
> tried it and I think it removes a lot of confusion.
>
>
>>
>>      def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
>>          '''yield a sequence of changegroup chunks (strings)'''
>> @@ -612,6 +626,7 @@ class cg1packer(object):
>>
>>          clrevorder = {}
>>          mfs = {} # needed manifests
>> +        tmfnodes = {}
>>          fnodes = {} # needed file nodes
>>          changedfiles = set()
>>
>> @@ -649,6 +664,11 @@ class cg1packer(object):
>>          # simply take the slowpath, which already has the 'clrevorder'
>> logic.
>>          # This was also fixed in cc0ff93d0c0c.
>>          fastpathlinkrev = fastpathlinkrev and not self._reorder
>> +        # Treemanifests don't work correctly with fastpathlinkrev
>> +        # either, because we don't discover which directory nodes to
>> +        # send along with files. This could probably be fixed.
>> +        fastpathlinkrev = fastpathlinkrev and (
>> +            'treemanifest' not in self._repo.requirements)
>>          # Callback for the manifest, used to collect linkrevs for filelog
>>          # revisions.
>>          # Returns the linkrev node (collected in lookupcl).
>> @@ -659,11 +679,14 @@ class cg1packer(object):
>>
>>              SIDE EFFECT:
>>
>> -              fclnodes gets populated with the list of relevant
>> -              file nodes if we're not using fastpathlinkrev.
>> +              1) fclnodes gets populated with the list of relevant
>> +                 file nodes if we're not using fastpathlinkrev
>> +              2) When treemanifests are in use, collects treemanifest
>> nodes
>> +                 to send
>>
>> -            Note that this means you can't trust fclnodes until
>> -            after manifests have been sent to the client.
>> +            Note that this means manifests must be completely sent to
>> +            the client before you can trust the list of files and
>> +            treemanifests to send.
>>              """
>>              clnode = mfs[x]
>>              if not fastpathlinkrev:
>> @@ -676,10 +699,26 @@ class cg1packer(object):
>>                          fclnode = fclnodes.setdefault(n, clnode)
>>                          if clrevorder[clnode] < clrevorder[fclnode]:
>>                              fclnodes[n] = clnode
>> +                        # gather list of changed treemanifest nodes
>> +                        if '/' in f and ('treemanifest' in
>> +                                         self._repo.requirements):
>> +                            dirs = f.split('/')[:-1]
>> +                            submf = mdata
>> +                            path = []
>> +                            for sd in dirs:
>> +                                submf = submf._dirs[sd + '/']
>>
>
> Using the "private" members in treemanifest directly is a little scary.
> The class takes care to load the manifest in all public methods, but
> accessing _dirs directly does not do that. I think it works here because
> the readfast() method called into readdelta(), which called into
> _slowreaddelta(), which populates the manifest with a diff. However, if the
> delta parent is not a parent, readfast() would not call readdelta(), it
> instead calls read(), which returns an uninitialized (lazily loaded)
> manifest. Besides the problem of it not being loaded (I *think*; we'd have
> to verify), it would be bad for narrowhg, which does not want the entire
> manifest iterated over.
>
> I think cg3packer.generate() should *always* call readdelta() for
> treemanifests (until we implement the fast path).
>
>
>> +                                fullpath = posixpath.join(*(path + [sd]))
>>
>
> I think fullpath can be replaced by submf.dir()
>
>
>> +                                path.append(sd)
>> +                                tmfclnodes = tmfnodes.setdefault(
>> +                                    fullpath, {})
>> +                                tmfclnodes.setdefault(submf.node(),
>> clnode)
>> +                                if clrevorder[clnode] <
>> clrevorder[fclnode]:
>> +                                    tmfclnodes[n] = clnode
>>              return clnode
>>
>>          mfnodes = self.prune(ml, mfs, commonrevs)
>> -        for x in self._packmanifests(mfnodes, lookupmflinknode):
>> +        for x in self._packmanifests(
>> +            mfnodes, tmfnodes, lookupmflinknode):
>>              yield x
>>
>>          mfs.clear()
>> @@ -797,9 +836,32 @@ class cg2packer(cg1packer):
>>      def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
>>          return struct.pack(self.deltaheader, node, p1n, p2n, basenode,
>> linknode)
>>
>> +class cg3packer(cg2packer):
>> +    version = '03'
>> +
>> +    def _packmanifests(self, mfnodes, tmfnodes, lookuplinknode):
>> +        # Note that debug prints are super confusing in this code, as
>> +        # tmfnodes gets populated by the calls to lookuplinknode in
>> +        # the superclass's manifest packer. In the future we should
>> +        # probably see if we can refactor this somehow to be less
>> +        # confusing.
>> +        for x in super(cg3packer, self)._packmanifests(
>> +            mfnodes, {}, lookuplinknode):
>> +            yield x
>> +        dirlog = self._repo.manifest.dirlog
>> +        for name, nodes in tmfnodes.iteritems():
>> +            # For now, directory headers are simply file headers with
>> +            # a trailing '/' on the path.
>> +            yield self.fileheader(name + '/')
>> +            for chunk in self.group(nodes, dirlog(name), nodes.get):
>> +                yield chunk
>> +
>> +
>>  packermap = {'01': (cg1packer, cg1unpacker),
>>               # cg2 adds support for exchanging generaldelta
>>               '02': (cg2packer, cg2unpacker),
>> +             # cg3 adds support for exchanging treemanifests
>> +             '03': (cg3packer, cg3unpacker),
>>  }
>>
>>  def _changegroupinfo(repo, nodes, source):
>> @@ -926,15 +988,33 @@ def _addchangegroupfiles(repo, source, r
>>          f = chunkdata["filename"]
>>          repo.ui.debug("adding %s revisions\n" % f)
>>          pr()
>> -        fl = repo.file(f)
>> +        directory = (f[-1] == '/')
>> +        if directory:
>> +            # a directory using treemanifests
>> +            # TODO fixup repo requirements safely
>> +            if 'treemanifest' not in repo.requirements:
>> +                if not wasempty:
>> +                    raise error.Abort(_(
>> +                        "bundle contains tree manifests, but local repo
>> is "
>> +                        "non-empty and does not use tree manifests"))
>> +                repo.requirements.add('treemanifest')
>> +                repo._applyopenerreqs()
>> +                repo._writerequirements()
>> +                repo.manifest._treeondisk = True
>> +                repo.manifest._treeinmem = True
>>
>
> This would not be necessary if the requirement had been added to the repo
> before the manifest revlog was accessed. How do we handle this elsewhere?
> Is this check what we would want the client to run into:
> https://selenic.com/hg/file/7e1fac6c0a9c/mercurial/exchange.py#l1103
>
>
>> +            fl = repo.manifest.dirlog(f)
>> +        else:
>> +            fl = repo.file(f)
>>          o = len(fl)
>>          try:
>>              if not fl.addgroup(source, revmap, trp):
>>                  raise error.Abort(_("received file revlog group is
>> empty"))
>>          except error.CensoredBaseError as e:
>>              raise error.Abort(_("received delta base is censored: %s") %
>> e)
>> -        revisions += len(fl) - o
>> -        files += 1
>> +        if not directory:
>> +            revisions += len(fl) - o
>> +            files += 1
>> +        # FIXME what is needfiles
>>          if f in needfiles:
>>              needs = needfiles[f]
>>              for new in xrange(o, len(fl)):
>> diff --git a/tests/test-acl.t b/tests/test-acl.t
>> --- a/tests/test-acl.t
>> +++ b/tests/test-acl.t
>> @@ -99,13 +99,13 @@ Extension disabled for lack of a hook
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -162,13 +162,13 @@ Extension disabled for lack of acl.sourc
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -228,13 +228,13 @@ No [acl.allow]/[acl.deny]
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -304,13 +304,13 @@ Empty [acl.allow]
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -369,13 +369,13 @@ fred is allowed inside foo/
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -439,13 +439,13 @@ Empty [acl.deny]
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -506,13 +506,13 @@ fred is allowed inside foo/, but not foo
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -578,13 +578,13 @@ fred is allowed inside foo/, but not foo
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -647,13 +647,13 @@ fred is allowed inside foo/, but not foo
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -718,13 +718,13 @@ barney is allowed everywhere
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -801,13 +801,13 @@ wilma can change files with a .txt exten
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -879,13 +879,13 @@ file specified by acl.config does not ex
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -952,13 +952,13 @@ betty is allowed inside foo/ by a acl.co
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1036,13 +1036,13 @@ acl.config can set only [acl.allow]/[acl
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1122,13 +1122,13 @@ fred is always allowed
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1204,13 +1204,13 @@ no one is allowed inside foo/Bar/
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1280,13 +1280,13 @@ OS-level groups
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1363,13 +1363,13 @@ OS-level groups
>>    f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    bundle2-output-bundle: "HG20", 4 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1478,14 +1478,14 @@ No branch acls specified
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1566,14 +1566,14 @@ Branch acl deny test
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1640,14 +1640,14 @@ Branch acl empty allow test
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1710,14 +1710,14 @@ Branch acl allow other
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1774,14 +1774,14 @@ Branch acl allow other
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1867,14 +1867,14 @@ push foobar into the remote
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -1959,14 +1959,14 @@ Branch acl conflicting deny
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -2028,14 +2028,14 @@ User 'astro' must not be denied
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> @@ -2114,14 +2114,14 @@ Non-astro users must be denied
>>    911600dab2ae7a9baff75958b84fe606851ce955
>>    e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
>>    bundle2-output-bundle: "HG20", 5 parts total
>> -  bundle2-output-part: "replycaps" 155 bytes payload
>> +  bundle2-output-part: "replycaps" 158 bytes payload
>>    bundle2-output-part: "check:heads" streamed payload
>>    bundle2-output-part: "changegroup" (params: 1 mandatory) streamed
>> payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
>>    bundle2-input-bundle: with-transaction
>>    bundle2-input-part: "replycaps" supported
>> -  bundle2-input-part: total payload size 155
>> +  bundle2-input-part: total payload size 158
>>    bundle2-input-part: "check:heads" supported
>>    bundle2-input-part: total payload size 20
>>    bundle2-input-part: "changegroup" (params: 1 mandatory) supported
>> diff --git a/tests/test-clonebundles.t b/tests/test-clonebundles.t
>> --- a/tests/test-clonebundles.t
>> +++ b/tests/test-clonebundles.t
>> @@ -33,7 +33,7 @@ Feature disabled by default
>>    $ cat server/access.log
>>    * - - [*] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
>>    * - - [*] "GET /?cmd=batch HTTP/1.1" 200 -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
>> -  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks
>> (glob)
>> +  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks
>> (glob)
>>    * - - [*] "GET /?cmd=listkeys HTTP/1.1" 200 -
>> x-hgarg-1:namespace=phases (glob)
>>
>>    $ cat >> $HGRCPATH << EOF
>> @@ -53,7 +53,7 @@ Missing manifest should not result in se
>>    $ tail -4 server/access.log
>>    * - - [*] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
>>    * - - [*] "GET /?cmd=batch HTTP/1.1" 200 -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
>> -  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks
>> (glob)
>> +  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks
>> (glob)
>>    * - - [*] "GET /?cmd=listkeys HTTP/1.1" 200 -
>> x-hgarg-1:namespace=phases (glob)
>>
>>  Empty manifest file results in retrieval
>> diff --git a/tests/test-hgweb-commands.t b/tests/test-hgweb-commands.t
>> --- a/tests/test-hgweb-commands.t
>> +++ b/tests/test-hgweb-commands.t
>> @@ -1897,7 +1897,7 @@ capabilities
>>    $ get-with-headers.py 127.0.0.1:$HGPORT '?cmd=capabilities'; echo
>>    200 Script output follows
>>
>> -  lookup changegroupsubset branchmap pushkey known getbundle
>> unbundlehash batch
>> bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1*%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 (glob)
>> +  lookup changegroupsubset branchmap pushkey known getbundle
>> unbundlehash batch
>> bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
>>
>>  heads
>>
>> @@ -2128,7 +2128,7 @@ capabilities
>>    batch
>>    stream-preferred
>>    streamreqs=generaldelta,revlogv1
>> -
>> bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>> +
>> bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>>    unbundle=HG10GZ,HG10BZ,HG10UN
>>    httpheader=1024
>>
>> diff --git a/tests/test-http-proxy.t b/tests/test-http-proxy.t
>> --- a/tests/test-http-proxy.t
>> +++ b/tests/test-http-proxy.t
>> @@ -110,21 +110,21 @@ do not use the proxy if it is in the no
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=branchmap HTTP/1.1" - -
>> (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=stream_out HTTP/1.1" - -
>> (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D83180e7845de420a1bb46896fd5fe05294f8d629
>> (glob)
>> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=83180e7845de420a1bb46896fd5fe05294f8d629&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=83180e7845de420a1bb46896fd5fe05294f8d629&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
>> x-hgarg-1:namespace=phases (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" -
>> - (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
>> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
>> x-hgarg-1:namespace=phases (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" -
>> - (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
>> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
>> x-hgarg-1:namespace=phases (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" -
>> - (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
>> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
>> x-hgarg-1:namespace=phases (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" -
>> - (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
>> -  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>> +  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks
>> (glob)
>>    * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - -
>> x-hgarg-1:namespace=phases (glob)
>> diff --git a/tests/test-http.t b/tests/test-http.t
>> --- a/tests/test-http.t
>> +++ b/tests/test-http.t
>> @@ -262,12 +262,12 @@ test http authentication
>>    "GET /?cmd=stream_out HTTP/1.1" 401 -
>>    "GET /?cmd=stream_out HTTP/1.1" 200 -
>>    "GET /?cmd=batch HTTP/1.1" 200 -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D5fed3813f7f5e1824344fdc9cf8f63bb662c292d
>> -  "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
>> +  "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
>>    "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
>>    "GET /?cmd=capabilities HTTP/1.1" 200 -
>>    "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D
>> -  "GET /?cmd=getbundle HTTP/1.1" 401 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
>> -  "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
>> +  "GET /?cmd=getbundle HTTP/1.1" 401 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
>> +  "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
>>    "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
>>    "GET /?cmd=capabilities HTTP/1.1" 200 -
>>    "GET /?cmd=lookup HTTP/1.1" 200 - x-hgarg-1:key=tip
>> diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t
>> --- a/tests/test-setdiscovery.t
>> +++ b/tests/test-setdiscovery.t
>> @@ -350,7 +350,7 @@ Test actual protocol when pulling one ne
>>    $ cut -d' ' -f6- access.log | grep -v cmd=known # cmd=known uses
>> random sampling
>>    "GET /?cmd=capabilities HTTP/1.1" 200 -
>>    "GET /?cmd=batch HTTP/1.1" 200 -
>> x-hgarg-1:cmds=heads+%3Bknown+nodes%3D513314ca8b3ae4dac8eec56966265b00fcf866db
>> -  "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=513314ca8b3ae4dac8eec56966265b00fcf866db&heads=e64a39e7da8b0d54bc63e81169aff001c13b3477
>> +  "GET /?cmd=getbundle HTTP/1.1" 200 -
>> x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=513314ca8b3ae4dac8eec56966265b00fcf866db&heads=e64a39e7da8b0d54bc63e81169aff001c13b3477
>>    "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
>>    $ cat errors.log
>>
>> diff --git a/tests/test-ssh-bundle1.t b/tests/test-ssh-bundle1.t
>> --- a/tests/test-ssh-bundle1.t
>> +++ b/tests/test-ssh-bundle1.t
>> @@ -462,8 +462,8 @@ debug output
>>    running python ".*/dummyssh" user@dummy ('|")hg -R remote serve
>> --stdio('|") (re)
>>    sending hello command
>>    sending between command
>> -  remote: 371
>> -  remote: capabilities: lookup changegroupsubset branchmap pushkey known
>> getbundle unbundlehash batch streamreqs=generaldelta,revlogv1
>> bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
>> +  remote: 376
>> +  remote: capabilities: lookup changegroupsubset branchmap pushkey known
>> getbundle unbundlehash batch streamreqs=generaldelta,revlogv1
>> bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
>>    remote: 1
>>    preparing listkeys for "bookmarks"
>>    sending listkeys command
>> diff --git a/tests/test-ssh.t b/tests/test-ssh.t
>> --- a/tests/test-ssh.t
>> +++ b/tests/test-ssh.t
>> @@ -454,8 +454,8 @@ debug output
>>    running python ".*/dummyssh" user@dummy ('|")hg -R remote serve
>> --stdio('|") (re)
>>    sending hello command
>>    sending between command
>> -  remote: 371
>> -  remote: capabilities: lookup changegroupsubset branchmap pushkey known
>> getbundle unbundlehash batch streamreqs=generaldelta,revlogv1
>> bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
>> +  remote: 376
>> +  remote: capabilities: lookup changegroupsubset branchmap pushkey known
>> getbundle unbundlehash batch streamreqs=generaldelta,revlogv1
>> bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
>> unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
>>    remote: 1
>>    query 1; heads
>>    sending batch command
>> diff --git a/tests/test-treemanifest.t b/tests/test-treemanifest.t
>> --- a/tests/test-treemanifest.t
>> +++ b/tests/test-treemanifest.t
>> @@ -388,3 +388,50 @@ within that.
>>    $ mv oldmf2 .hg/store/meta/b/foo
>>    $ mv oldmf3 .hg/store/meta/b/bar/orange
>>
>> +Test cloning a treemanifest repo over http.
>> +  $ hg serve -p $HGPORT -d --pid-file=hg.pid --errorlog=errors.log
>> +  $ cat hg.pid >> $DAEMON_PIDS
>> +  $ cd ..
>> +We can clone even with the knob turned off and we'll get a treemanifest
>> repo.
>> +  $ hg clone --config experimental.treemanifest=False \
>> +  >   http://localhost:$HGPORT deepclone
>> +  requesting all changes
>> +  adding changesets
>> +  adding manifests
>> +  adding file changes
>> +  added 1 changesets with 8 changes to 8 files
>> +  updating to branch default
>> +  8 files updated, 0 files merged, 0 files removed, 0 files unresolved
>> +No server errors.
>> +  $ cat deeprepo/errors.log
>> +requires got updated to include treemanifest
>> +  $ cat deepclone/.hg/requires | grep treemanifest
>> +  treemanifest
>> +Tree manifest revlogs exist.
>> +  $ find deepclone/.hg/store/meta | sort
>> +  deepclone/.hg/store/meta
>> +  deepclone/.hg/store/meta/a
>> +  deepclone/.hg/store/meta/a/00manifest.i
>> +  deepclone/.hg/store/meta/b
>> +  deepclone/.hg/store/meta/b/00manifest.i
>> +  deepclone/.hg/store/meta/b/bar
>> +  deepclone/.hg/store/meta/b/bar/00manifest.i
>> +  deepclone/.hg/store/meta/b/bar/orange
>> +  deepclone/.hg/store/meta/b/bar/orange/00manifest.i
>> +  deepclone/.hg/store/meta/b/bar/orange/fly
>> +  deepclone/.hg/store/meta/b/bar/orange/fly/00manifest.i
>> +  deepclone/.hg/store/meta/b/foo
>> +  deepclone/.hg/store/meta/b/foo/00manifest.i
>> +  deepclone/.hg/store/meta/b/foo/apple
>> +  deepclone/.hg/store/meta/b/foo/apple/00manifest.i
>> +  deepclone/.hg/store/meta/b/foo/apple/bees
>> +  deepclone/.hg/store/meta/b/foo/apple/bees/00manifest.i
>> +Verify passes.
>> +  $ cd deepclone
>> +  $ hg verify
>> +  checking changesets
>> +  checking manifests
>> +  crosschecking files in changesets and manifests
>> +  checking files
>> +  8 files, 1 changesets, 8 total revisions
>> +  $ cd ..
>>
>
Augie Fackler - Dec. 4, 2015, 12:04 a.m.
(responses inline, tried to prune a fair amount to keep it easier to follow. I’ll hold off on a v2 until my questions below are resolved. Thanks for the review!)

> On Dec 3, 2015, at 12:42 PM, Martin von Zweigbergk <martinvonz@google.com> wrote:
> 
> 
> 
> On Thu, Dec 3, 2015 at 7:58 AM Augie Fackler <raf@durin42.com> wrote:
>> # HG changeset patch
>> # User Augie Fackler <augie@google.com>
>> # Date 1449092255 21600
>> #      Wed Dec 02 15:37:35 2015 -0600
>> # Node ID 1240273d0971ff1d5e596ffea69cf34a07335d92
>> # Parent  79ad570337631038a08b75957b2890975221e089

[…]

>> @@ -603,6 +612,11 @@ class cg1packer(object):
>>              size += len(chunk)
>>              yield chunk
>>          self._verbosenote(_('%8.i (manifests)\n') % size)
>> +        # It looks odd to assert this here, but tmfnodes doesn't get
>> +        # filled in until after we've called lookuplinknode for
>> +        # sending root manifests, so the only way to tell the streams
>> +        # got crossed is to check after we've done all the work.
>> +        assert not tmfnodes
>> 
> I think it would be clearer to override the entire generate() method and add call the cg2packer.generate() from there if not using tree manifests and then dedicate the rest of the method to tree manifests. It would mean more code duplication, but some of that could be extracted to helpers. I tried it and I think it removes a lot of confusion.

I get where you’re coming from, but generate is a place where we’re likely to do performance work in the future, and I’d like to keep it to one (slightly more complicated) implementation, rather than two that we have to remember to update in parallel.

>> 
>> @@ -676,10 +699,26 @@ class cg1packer(object):
>>                          fclnode = fclnodes.setdefault(n, clnode)
>>                          if clrevorder[clnode] < clrevorder[fclnode]:
>>                              fclnodes[n] = clnode
>> +                        # gather list of changed treemanifest nodes
>> +                        if '/' in f and ('treemanifest' in
>> +                                         self._repo.requirements):
>> +                            dirs = f.split('/')[:-1]
>> +                            submf = mdata
>> +                            path = []
>> +                            for sd in dirs:
>> +                                submf = submf._dirs[sd + '/']
> 
> Using the "private" members in treemanifest directly is a little scary. The class takes care to load the manifest in all public methods, but accessing _dirs directly does not do that. I think it works here because the readfast() method called into readdelta(), which called into _slowreaddelta(), which populates the manifest with a diff. However, if the delta parent is not a parent, readfast() would not call readdelta(), it instead calls read(), which returns an uninitialized (lazily loaded) manifest. Besides the problem of it not being loaded (I *think*; we'd have to verify), it would be bad for narrowhg, which does not want the entire manifest iterated over.

I agree, I’d rather not use the private method, but do I have a choice? How should I ensure the submf is loaded?

(I think the submf sure to already be fully loaded because immediately before we do the directory checks we looked at a file along the same path, so I’m not sure we need any extra paranoia today. Should I add a comment to that effect?)

> 
> I think cg3packer.generate() should *always* call readdelta() for treemanifests (until we implement the fast path).

I don’t follow this at all. :(


[snip]

> +            if 'treemanifest' not in repo.requirements:
> +                if not wasempty:
> +                    raise error.Abort(_(
> +                        "bundle contains tree manifests, but local repo is "
> +                        "non-empty and does not use tree manifests"))
> +                repo.requirements.add('treemanifest')
> +                repo._applyopenerreqs()
> +                repo._writerequirements()
> +                repo.manifest._treeondisk = True
> +                repo.manifest._treeinmem = True
> 
> This would not be necessary if the requirement had been added to the repo before the manifest revlog was accessed. How do we handle this elsewhere? Is this check what we would want the client to run into: https://selenic.com/hg/file/7e1fac6c0a9c/mercurial/exchange.py#l1103

It’s weird. Basically, clone creates a repo on-disk and then does a pull into it, so it doesn’t turn on tree manifest as a requirement because it doesn’t know until it’s too late. Does that make sense?
Augie Fackler - Dec. 4, 2015, 1:16 a.m.
For the benefit of folks on the list: Martin and I chatted via VC a bit ago, and I expect to mail a v2 soon, probably tomorrow.

AF

> On Dec 3, 2015, at 7:04 PM, Augie Fackler <raf@durin42.com> wrote:
> 
> (responses inline, tried to prune a fair amount to keep it easier to follow. I’ll hold off on a v2 until my questions below are resolved. Thanks for the review!)
> 
>> On Dec 3, 2015, at 12:42 PM, Martin von Zweigbergk <martinvonz@google.com> wrote:
>> 
>> 
>> 
>> On Thu, Dec 3, 2015 at 7:58 AM Augie Fackler <raf@durin42.com> wrote:
>>> # HG changeset patch
>>> # User Augie Fackler <augie@google.com>
>>> # Date 1449092255 21600
>>> #      Wed Dec 02 15:37:35 2015 -0600
>>> # Node ID 1240273d0971ff1d5e596ffea69cf34a07335d92
>>> # Parent  79ad570337631038a08b75957b2890975221e089
> 
> […]
> 
>>> @@ -603,6 +612,11 @@ class cg1packer(object):
>>>             size += len(chunk)
>>>             yield chunk
>>>         self._verbosenote(_('%8.i (manifests)\n') % size)
>>> +        # It looks odd to assert this here, but tmfnodes doesn't get
>>> +        # filled in until after we've called lookuplinknode for
>>> +        # sending root manifests, so the only way to tell the streams
>>> +        # got crossed is to check after we've done all the work.
>>> +        assert not tmfnodes
>>> 
>> I think it would be clearer to override the entire generate() method and add call the cg2packer.generate() from there if not using tree manifests and then dedicate the rest of the method to tree manifests. It would mean more code duplication, but some of that could be extracted to helpers. I tried it and I think it removes a lot of confusion.
> 
> I get where you’re coming from, but generate is a place where we’re likely to do performance work in the future, and I’d like to keep it to one (slightly more complicated) implementation, rather than two that we have to remember to update in parallel.
> 
>>> 
>>> @@ -676,10 +699,26 @@ class cg1packer(object):
>>>                         fclnode = fclnodes.setdefault(n, clnode)
>>>                         if clrevorder[clnode] < clrevorder[fclnode]:
>>>                             fclnodes[n] = clnode
>>> +                        # gather list of changed treemanifest nodes
>>> +                        if '/' in f and ('treemanifest' in
>>> +                                         self._repo.requirements):
>>> +                            dirs = f.split('/')[:-1]
>>> +                            submf = mdata
>>> +                            path = []
>>> +                            for sd in dirs:
>>> +                                submf = submf._dirs[sd + '/']
>> 
>> Using the "private" members in treemanifest directly is a little scary. The class takes care to load the manifest in all public methods, but accessing _dirs directly does not do that. I think it works here because the readfast() method called into readdelta(), which called into _slowreaddelta(), which populates the manifest with a diff. However, if the delta parent is not a parent, readfast() would not call readdelta(), it instead calls read(), which returns an uninitialized (lazily loaded) manifest. Besides the problem of it not being loaded (I *think*; we'd have to verify), it would be bad for narrowhg, which does not want the entire manifest iterated over.
> 
> I agree, I’d rather not use the private method, but do I have a choice? How should I ensure the submf is loaded?
> 
> (I think the submf sure to already be fully loaded because immediately before we do the directory checks we looked at a file along the same path, so I’m not sure we need any extra paranoia today. Should I add a comment to that effect?)
> 
>> 
>> I think cg3packer.generate() should *always* call readdelta() for treemanifests (until we implement the fast path).
> 
> I don’t follow this at all. :(
> 
> 
> [snip]
> 
>> +            if 'treemanifest' not in repo.requirements:
>> +                if not wasempty:
>> +                    raise error.Abort(_(
>> +                        "bundle contains tree manifests, but local repo is "
>> +                        "non-empty and does not use tree manifests"))
>> +                repo.requirements.add('treemanifest')
>> +                repo._applyopenerreqs()
>> +                repo._writerequirements()
>> +                repo.manifest._treeondisk = True
>> +                repo.manifest._treeinmem = True
>> 
>> This would not be necessary if the requirement had been added to the repo before the manifest revlog was accessed. How do we handle this elsewhere? Is this check what we would want the client to run into: https://selenic.com/hg/file/7e1fac6c0a9c/mercurial/exchange.py#l1103
> 
> It’s weird. Basically, clone creates a repo on-disk and then does a pull into it, so it doesn’t turn on tree manifest as a requirement because it doesn’t know until it’s too late. Does that make sense?
> 
>
Martin von Zweigbergk - Dec. 4, 2015, 6:23 p.m.
On Thu, Dec 3, 2015 at 4:04 PM Augie Fackler <raf@durin42.com> wrote:

> (responses inline, tried to prune a fair amount to keep it easier to
> follow. I’ll hold off on a v2 until my questions below are resolved. Thanks
> for the review!)
>
> > On Dec 3, 2015, at 12:42 PM, Martin von Zweigbergk <
> martinvonz@google.com> wrote:
> >
> >
> >
> > On Thu, Dec 3, 2015 at 7:58 AM Augie Fackler <raf@durin42.com> wrote:
> >> # HG changeset patch
> >> # User Augie Fackler <augie@google.com>
> >> # Date 1449092255 21600
> >> #      Wed Dec 02 15:37:35 2015 -0600
> >> # Node ID 1240273d0971ff1d5e596ffea69cf34a07335d92
> >> # Parent  79ad570337631038a08b75957b2890975221e089
>
> […]
>
> >> @@ -603,6 +612,11 @@ class cg1packer(object):
> >>              size += len(chunk)
> >>              yield chunk
> >>          self._verbosenote(_('%8.i (manifests)\n') % size)
> >> +        # It looks odd to assert this here, but tmfnodes doesn't get
> >> +        # filled in until after we've called lookuplinknode for
> >> +        # sending root manifests, so the only way to tell the streams
> >> +        # got crossed is to check after we've done all the work.
> >> +        assert not tmfnodes
> >>
> > I think it would be clearer to override the entire generate() method and
> add call the cg2packer.generate() from there if not using tree manifests
> and then dedicate the rest of the method to tree manifests. It would mean
> more code duplication, but some of that could be extracted to helpers. I
> tried it and I think it removes a lot of confusion.
>
> I get where you’re coming from, but generate is a place where we’re likely
> to do performance work in the future, and I’d like to keep it to one
> (slightly more complicated) implementation, rather than two that we have to
> remember to update in parallel.
>




>
> >>
> >> @@ -676,10 +699,26 @@ class cg1packer(object):
> >>                          fclnode = fclnodes.setdefault(n, clnode)
> >>                          if clrevorder[clnode] < clrevorder[fclnode]:
> >>                              fclnodes[n] = clnode
> >> +                        # gather list of changed treemanifest nodes
> >> +                        if '/' in f and ('treemanifest' in
> >> +                                         self._repo.requirements):
> >> +                            dirs = f.split('/')[:-1]
> >> +                            submf = mdata
> >> +                            path = []
> >> +                            for sd in dirs:
> >> +                                submf = submf._dirs[sd + '/']
> >
> > Using the "private" members in treemanifest directly is a little scary.
> The class takes care to load the manifest in all public methods, but
> accessing _dirs directly does not do that. I think it works here because
> the readfast() method called into readdelta(), which called into
> _slowreaddelta(), which populates the manifest with a diff. However, if the
> delta parent is not a parent, readfast() would not call readdelta(), it
> instead calls read(), which returns an uninitialized (lazily loaded)
> manifest. Besides the problem of it not being loaded (I *think*; we'd have
> to verify), it would be bad for narrowhg, which does not want the entire
> manifest iterated over.
>
> I agree, I’d rather not use the private method, but do I have a choice?
> How should I ensure the submf is loaded?
>
> (I think the submf sure to already be fully loaded because immediately
> before we do the directory checks we looked at a file along the same path,
> so I’m not sure we need any extra paranoia today. Should I add a comment to
> that effect?)
>

Yes, you're right, the iteration does mean that they are loaded. And I was
wrong about it calling readdelta(): it actually calls read(), and if it had
called readdelta(), it would fail on submf.node(). So if you keep the call
to readfast() for now, you should replace the .node() by ._node. I think we
should include more than one changeset in the clone test to make sure
readfast() results in both read() and readdelta().


>
> >
> > I think cg3packer.generate() should *always* call readdelta() for
> treemanifests (until we implement the fast path).
>
> I don’t follow this at all. :(
>

It might not have made sense. My goal was to avoid mf.read().iteritems()
since that is bad for narrowhg (and narrowhg is the main reason for tree
manifests). Anyway, we talked about this on VC, and it can be improved
later.



>
>
> [snip]
>
> > +            if 'treemanifest' not in repo.requirements:
> > +                if not wasempty:
> > +                    raise error.Abort(_(
> > +                        "bundle contains tree manifests, but local repo
> is "
> > +                        "non-empty and does not use tree manifests"))
> > +                repo.requirements.add('treemanifest')
> > +                repo._applyopenerreqs()
> > +                repo._writerequirements()
> > +                repo.manifest._treeondisk = True
> > +                repo.manifest._treeinmem = True
> >
> > This would not be necessary if the requirement had been added to the
> repo before the manifest revlog was accessed. How do we handle this
> elsewhere? Is this check what we would want the client to run into:
> https://selenic.com/hg/file/7e1fac6c0a9c/mercurial/exchange.py#l1103
>
> It’s weird. Basically, clone creates a repo on-disk and then does a pull
> into it, so it doesn’t turn on tree manifest as a requirement because it
> doesn’t know until it’s too late. Does that make sense?
>
>
>

Patch

diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -8,6 +8,7 @@ 
 from __future__ import absolute_import
 
 import os
+import posixpath
 import struct
 import tempfile
 import weakref
@@ -497,6 +498,14 @@  class cg2unpacker(cg1unpacker):
         node, p1, p2, deltabase, cs = headertuple
         return node, p1, p2, deltabase, cs
 
+class cg3unpacker(cg2unpacker):
+    """Unpacker for cg3 streams.
+
+    cg3 streams add support for exchanging treemanifests, so the only
+    thing that changes is the version number.
+    """
+    version = '03'
+
 class headerlessfixup(object):
     def __init__(self, fh, h):
         self._h = h
@@ -594,7 +603,7 @@  class cg1packer(object):
         rr, rl = revlog.rev, revlog.linkrev
         return [n for n in missing if rl(rr(n)) not in commonrevs]
 
-    def _packmanifests(self, mfnodes, lookuplinknode):
+    def _packmanifests(self, mfnodes, tmfnodes, lookuplinknode):
         """Pack flat manifests into a changegroup stream."""
         ml = self._repo.manifest
         size = 0
@@ -603,6 +612,11 @@  class cg1packer(object):
             size += len(chunk)
             yield chunk
         self._verbosenote(_('%8.i (manifests)\n') % size)
+        # It looks odd to assert this here, but tmfnodes doesn't get
+        # filled in until after we've called lookuplinknode for
+        # sending root manifests, so the only way to tell the streams
+        # got crossed is to check after we've done all the work.
+        assert not tmfnodes
 
     def generate(self, commonrevs, clnodes, fastpathlinkrev, source):
         '''yield a sequence of changegroup chunks (strings)'''
@@ -612,6 +626,7 @@  class cg1packer(object):
 
         clrevorder = {}
         mfs = {} # needed manifests
+        tmfnodes = {}
         fnodes = {} # needed file nodes
         changedfiles = set()
 
@@ -649,6 +664,11 @@  class cg1packer(object):
         # simply take the slowpath, which already has the 'clrevorder' logic.
         # This was also fixed in cc0ff93d0c0c.
         fastpathlinkrev = fastpathlinkrev and not self._reorder
+        # Treemanifests don't work correctly with fastpathlinkrev
+        # either, because we don't discover which directory nodes to
+        # send along with files. This could probably be fixed.
+        fastpathlinkrev = fastpathlinkrev and (
+            'treemanifest' not in self._repo.requirements)
         # Callback for the manifest, used to collect linkrevs for filelog
         # revisions.
         # Returns the linkrev node (collected in lookupcl).
@@ -659,11 +679,14 @@  class cg1packer(object):
 
             SIDE EFFECT:
 
-              fclnodes gets populated with the list of relevant
-              file nodes if we're not using fastpathlinkrev.
+              1) fclnodes gets populated with the list of relevant
+                 file nodes if we're not using fastpathlinkrev
+              2) When treemanifests are in use, collects treemanifest nodes
+                 to send
 
-            Note that this means you can't trust fclnodes until
-            after manifests have been sent to the client.
+            Note that this means manifests must be completely sent to
+            the client before you can trust the list of files and
+            treemanifests to send.
             """
             clnode = mfs[x]
             if not fastpathlinkrev:
@@ -676,10 +699,26 @@  class cg1packer(object):
                         fclnode = fclnodes.setdefault(n, clnode)
                         if clrevorder[clnode] < clrevorder[fclnode]:
                             fclnodes[n] = clnode
+                        # gather list of changed treemanifest nodes
+                        if '/' in f and ('treemanifest' in
+                                         self._repo.requirements):
+                            dirs = f.split('/')[:-1]
+                            submf = mdata
+                            path = []
+                            for sd in dirs:
+                                submf = submf._dirs[sd + '/']
+                                fullpath = posixpath.join(*(path + [sd]))
+                                path.append(sd)
+                                tmfclnodes = tmfnodes.setdefault(
+                                    fullpath, {})
+                                tmfclnodes.setdefault(submf.node(), clnode)
+                                if clrevorder[clnode] < clrevorder[fclnode]:
+                                    tmfclnodes[n] = clnode
             return clnode
 
         mfnodes = self.prune(ml, mfs, commonrevs)
-        for x in self._packmanifests(mfnodes, lookupmflinknode):
+        for x in self._packmanifests(
+            mfnodes, tmfnodes, lookupmflinknode):
             yield x
 
         mfs.clear()
@@ -797,9 +836,32 @@  class cg2packer(cg1packer):
     def builddeltaheader(self, node, p1n, p2n, basenode, linknode):
         return struct.pack(self.deltaheader, node, p1n, p2n, basenode, linknode)
 
+class cg3packer(cg2packer):
+    version = '03'
+
+    def _packmanifests(self, mfnodes, tmfnodes, lookuplinknode):
+        # Note that debug prints are super confusing in this code, as
+        # tmfnodes gets populated by the calls to lookuplinknode in
+        # the superclass's manifest packer. In the future we should
+        # probably see if we can refactor this somehow to be less
+        # confusing.
+        for x in super(cg3packer, self)._packmanifests(
+            mfnodes, {}, lookuplinknode):
+            yield x
+        dirlog = self._repo.manifest.dirlog
+        for name, nodes in tmfnodes.iteritems():
+            # For now, directory headers are simply file headers with
+            # a trailing '/' on the path.
+            yield self.fileheader(name + '/')
+            for chunk in self.group(nodes, dirlog(name), nodes.get):
+                yield chunk
+
+
 packermap = {'01': (cg1packer, cg1unpacker),
              # cg2 adds support for exchanging generaldelta
              '02': (cg2packer, cg2unpacker),
+             # cg3 adds support for exchanging treemanifests
+             '03': (cg3packer, cg3unpacker),
 }
 
 def _changegroupinfo(repo, nodes, source):
@@ -926,15 +988,33 @@  def _addchangegroupfiles(repo, source, r
         f = chunkdata["filename"]
         repo.ui.debug("adding %s revisions\n" % f)
         pr()
-        fl = repo.file(f)
+        directory = (f[-1] == '/')
+        if directory:
+            # a directory using treemanifests
+            # TODO fixup repo requirements safely
+            if 'treemanifest' not in repo.requirements:
+                if not wasempty:
+                    raise error.Abort(_(
+                        "bundle contains tree manifests, but local repo is "
+                        "non-empty and does not use tree manifests"))
+                repo.requirements.add('treemanifest')
+                repo._applyopenerreqs()
+                repo._writerequirements()
+                repo.manifest._treeondisk = True
+                repo.manifest._treeinmem = True
+            fl = repo.manifest.dirlog(f)
+        else:
+            fl = repo.file(f)
         o = len(fl)
         try:
             if not fl.addgroup(source, revmap, trp):
                 raise error.Abort(_("received file revlog group is empty"))
         except error.CensoredBaseError as e:
             raise error.Abort(_("received delta base is censored: %s") % e)
-        revisions += len(fl) - o
-        files += 1
+        if not directory:
+            revisions += len(fl) - o
+            files += 1
+        # FIXME what is needfiles
         if f in needfiles:
             needs = needfiles[f]
             for new in xrange(o, len(fl)):
diff --git a/tests/test-acl.t b/tests/test-acl.t
--- a/tests/test-acl.t
+++ b/tests/test-acl.t
@@ -99,13 +99,13 @@  Extension disabled for lack of a hook
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -162,13 +162,13 @@  Extension disabled for lack of acl.sourc
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -228,13 +228,13 @@  No [acl.allow]/[acl.deny]
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -304,13 +304,13 @@  Empty [acl.allow]
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -369,13 +369,13 @@  fred is allowed inside foo/
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -439,13 +439,13 @@  Empty [acl.deny]
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -506,13 +506,13 @@  fred is allowed inside foo/, but not foo
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -578,13 +578,13 @@  fred is allowed inside foo/, but not foo
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -647,13 +647,13 @@  fred is allowed inside foo/, but not foo
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -718,13 +718,13 @@  barney is allowed everywhere
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -801,13 +801,13 @@  wilma can change files with a .txt exten
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -879,13 +879,13 @@  file specified by acl.config does not ex
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -952,13 +952,13 @@  betty is allowed inside foo/ by a acl.co
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1036,13 +1036,13 @@  acl.config can set only [acl.allow]/[acl
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1122,13 +1122,13 @@  fred is always allowed
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1204,13 +1204,13 @@  no one is allowed inside foo/Bar/
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1280,13 +1280,13 @@  OS-level groups
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1363,13 +1363,13 @@  OS-level groups
   f9cafe1212c8c6fa1120d14a556e18cc44ff8bdd
   911600dab2ae7a9baff75958b84fe606851ce955
   bundle2-output-bundle: "HG20", 4 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1478,14 +1478,14 @@  No branch acls specified
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1566,14 +1566,14 @@  Branch acl deny test
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1640,14 +1640,14 @@  Branch acl empty allow test
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1710,14 +1710,14 @@  Branch acl allow other
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1774,14 +1774,14 @@  Branch acl allow other
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1867,14 +1867,14 @@  push foobar into the remote
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -1959,14 +1959,14 @@  Branch acl conflicting deny
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -2028,14 +2028,14 @@  User 'astro' must not be denied
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
@@ -2114,14 +2114,14 @@  Non-astro users must be denied
   911600dab2ae7a9baff75958b84fe606851ce955
   e8fc755d4d8217ee5b0c2bb41558c40d43b92c01
   bundle2-output-bundle: "HG20", 5 parts total
-  bundle2-output-part: "replycaps" 155 bytes payload
+  bundle2-output-part: "replycaps" 158 bytes payload
   bundle2-output-part: "check:heads" streamed payload
   bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "replycaps" supported
-  bundle2-input-part: total payload size 155
+  bundle2-input-part: total payload size 158
   bundle2-input-part: "check:heads" supported
   bundle2-input-part: total payload size 20
   bundle2-input-part: "changegroup" (params: 1 mandatory) supported
diff --git a/tests/test-clonebundles.t b/tests/test-clonebundles.t
--- a/tests/test-clonebundles.t
+++ b/tests/test-clonebundles.t
@@ -33,7 +33,7 @@  Feature disabled by default
   $ cat server/access.log
   * - - [*] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
   * - - [*] "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
-  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks (glob)
+  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks (glob)
   * - - [*] "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases (glob)
 
   $ cat >> $HGRCPATH << EOF
@@ -53,7 +53,7 @@  Missing manifest should not result in se
   $ tail -4 server/access.log
   * - - [*] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
   * - - [*] "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
-  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks (glob)
+  * - - [*] "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=aaff8d2ffbbf07a46dd1f05d8ae7877e3f56e2a2&listkeys=phase%2Cbookmarks (glob)
   * - - [*] "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases (glob)
 
 Empty manifest file results in retrieval
diff --git a/tests/test-hgweb-commands.t b/tests/test-hgweb-commands.t
--- a/tests/test-hgweb-commands.t
+++ b/tests/test-hgweb-commands.t
@@ -1897,7 +1897,7 @@  capabilities
   $ get-with-headers.py 127.0.0.1:$HGPORT '?cmd=capabilities'; echo
   200 Script output follows
   
-  lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1*%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024 (glob)
+  lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
 
 heads
 
@@ -2128,7 +2128,7 @@  capabilities
   batch
   stream-preferred
   streamreqs=generaldelta,revlogv1
-  bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
+  bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps
   unbundle=HG10GZ,HG10BZ,HG10UN
   httpheader=1024
 
diff --git a/tests/test-http-proxy.t b/tests/test-http-proxy.t
--- a/tests/test-http-proxy.t
+++ b/tests/test-http-proxy.t
@@ -110,21 +110,21 @@  do not use the proxy if it is in the no 
   * - - [*] "GET http://localhost:$HGPORT/?cmd=branchmap HTTP/1.1" - - (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=stream_out HTTP/1.1" - - (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D83180e7845de420a1bb46896fd5fe05294f8d629 (glob)
-  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=83180e7845de420a1bb46896fd5fe05294f8d629&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
+  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=83180e7845de420a1bb46896fd5fe05294f8d629&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - - x-hgarg-1:namespace=phases (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" - - (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
-  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
+  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - - x-hgarg-1:namespace=phases (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" - - (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
-  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
+  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - - x-hgarg-1:namespace=phases (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" - - (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
-  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
+  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - - x-hgarg-1:namespace=phases (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" - - (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D (glob)
-  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
+  * - - [*] "GET http://localhost:$HGPORT/?cmd=getbundle HTTP/1.1" - - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=83180e7845de420a1bb46896fd5fe05294f8d629&listkeys=phase%2Cbookmarks (glob)
   * - - [*] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - - x-hgarg-1:namespace=phases (glob)
diff --git a/tests/test-http.t b/tests/test-http.t
--- a/tests/test-http.t
+++ b/tests/test-http.t
@@ -262,12 +262,12 @@  test http authentication
   "GET /?cmd=stream_out HTTP/1.1" 401 -
   "GET /?cmd=stream_out HTTP/1.1" 200 -
   "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D5fed3813f7f5e1824344fdc9cf8f63bb662c292d
-  "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
+  "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=0&common=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
   "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
   "GET /?cmd=capabilities HTTP/1.1" 200 -
   "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D
-  "GET /?cmd=getbundle HTTP/1.1" 401 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
-  "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
+  "GET /?cmd=getbundle HTTP/1.1" 401 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
+  "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=0000000000000000000000000000000000000000&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=phase%2Cbookmarks
   "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
   "GET /?cmd=capabilities HTTP/1.1" 200 -
   "GET /?cmd=lookup HTTP/1.1" 200 - x-hgarg-1:key=tip
diff --git a/tests/test-setdiscovery.t b/tests/test-setdiscovery.t
--- a/tests/test-setdiscovery.t
+++ b/tests/test-setdiscovery.t
@@ -350,7 +350,7 @@  Test actual protocol when pulling one ne
   $ cut -d' ' -f6- access.log | grep -v cmd=known # cmd=known uses random sampling
   "GET /?cmd=capabilities HTTP/1.1" 200 -
   "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D513314ca8b3ae4dac8eec56966265b00fcf866db
-  "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=513314ca8b3ae4dac8eec56966265b00fcf866db&heads=e64a39e7da8b0d54bc63e81169aff001c13b3477
+  "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bundlecaps=HG20%2Cbundle2%3DHG20%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps&cg=1&common=513314ca8b3ae4dac8eec56966265b00fcf866db&heads=e64a39e7da8b0d54bc63e81169aff001c13b3477
   "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases
   $ cat errors.log
 
diff --git a/tests/test-ssh-bundle1.t b/tests/test-ssh-bundle1.t
--- a/tests/test-ssh-bundle1.t
+++ b/tests/test-ssh-bundle1.t
@@ -462,8 +462,8 @@  debug output
   running python ".*/dummyssh" user@dummy ('|")hg -R remote serve --stdio('|") (re)
   sending hello command
   sending between command
-  remote: 371
-  remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
+  remote: 376
+  remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
   remote: 1
   preparing listkeys for "bookmarks"
   sending listkeys command
diff --git a/tests/test-ssh.t b/tests/test-ssh.t
--- a/tests/test-ssh.t
+++ b/tests/test-ssh.t
@@ -454,8 +454,8 @@  debug output
   running python ".*/dummyssh" user@dummy ('|")hg -R remote serve --stdio('|") (re)
   sending hello command
   sending between command
-  remote: 371
-  remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 bundle2=HG20%0Achangegroup%3D01%2C02%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
+  remote: 376
+  remote: capabilities: lookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch streamreqs=generaldelta,revlogv1 bundle2=HG20%0Achangegroup%3D01%2C02%2C03%0Adigests%3Dmd5%2Csha1%2Csha512%0Aerror%3Dabort%2Cunsupportedcontent%2Cpushraced%2Cpushkey%0Ahgtagsfnodes%0Alistkeys%0Apushkey%0Aremote-changegroup%3Dhttp%2Chttps unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024
   remote: 1
   query 1; heads
   sending batch command
diff --git a/tests/test-treemanifest.t b/tests/test-treemanifest.t
--- a/tests/test-treemanifest.t
+++ b/tests/test-treemanifest.t
@@ -388,3 +388,50 @@  within that.
   $ mv oldmf2 .hg/store/meta/b/foo
   $ mv oldmf3 .hg/store/meta/b/bar/orange
 
+Test cloning a treemanifest repo over http.
+  $ hg serve -p $HGPORT -d --pid-file=hg.pid --errorlog=errors.log
+  $ cat hg.pid >> $DAEMON_PIDS
+  $ cd ..
+We can clone even with the knob turned off and we'll get a treemanifest repo.
+  $ hg clone --config experimental.treemanifest=False \
+  >   http://localhost:$HGPORT deepclone
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 8 changes to 8 files
+  updating to branch default
+  8 files updated, 0 files merged, 0 files removed, 0 files unresolved
+No server errors.
+  $ cat deeprepo/errors.log
+requires got updated to include treemanifest
+  $ cat deepclone/.hg/requires | grep treemanifest
+  treemanifest
+Tree manifest revlogs exist.
+  $ find deepclone/.hg/store/meta | sort
+  deepclone/.hg/store/meta
+  deepclone/.hg/store/meta/a
+  deepclone/.hg/store/meta/a/00manifest.i
+  deepclone/.hg/store/meta/b
+  deepclone/.hg/store/meta/b/00manifest.i
+  deepclone/.hg/store/meta/b/bar
+  deepclone/.hg/store/meta/b/bar/00manifest.i
+  deepclone/.hg/store/meta/b/bar/orange
+  deepclone/.hg/store/meta/b/bar/orange/00manifest.i
+  deepclone/.hg/store/meta/b/bar/orange/fly
+  deepclone/.hg/store/meta/b/bar/orange/fly/00manifest.i
+  deepclone/.hg/store/meta/b/foo
+  deepclone/.hg/store/meta/b/foo/00manifest.i
+  deepclone/.hg/store/meta/b/foo/apple
+  deepclone/.hg/store/meta/b/foo/apple/00manifest.i
+  deepclone/.hg/store/meta/b/foo/apple/bees
+  deepclone/.hg/store/meta/b/foo/apple/bees/00manifest.i
+Verify passes.
+  $ cd deepclone
+  $ hg verify
+  checking changesets
+  checking manifests
+  crosschecking files in changesets and manifests
+  checking files
+  8 files, 1 changesets, 8 total revisions
+  $ cd ..