Patchwork [2,of,2] changegroup: skip delta when the underlying revlog do not use them

login
register
mail settings
Submitter Pierre-Yves David
Date Oct. 14, 2016, 1:08 a.m.
Message ID <747c0a1084ef6251a987.1476407286@nodosa.octopoid.net>
Download mbox | patch
Permalink /patch/17073/
State Accepted
Headers show

Comments

Pierre-Yves David - Oct. 14, 2016, 1:08 a.m.
# HG changeset patch
# User Pierre-Yves David <pierre-yves.david@ens-lyon.org>
# Date 1476401471 -7200
#      Fri Oct 14 01:31:11 2016 +0200
# Node ID 747c0a1084ef6251a987e469197bad7796756403
# Parent  e19eb107706e7210c3b359d66f5274911b181566
# EXP-Topic storedeltachains
changegroup: skip delta when the underlying revlog do not use them

Revlog can now be configured to store full snapshot only. This is used on the
changelog. However, the changegroup packing was still recomputing deltas to be
sent over the wire.

We now just reuse the full snapshot directly in this case, skipping delta
computation. This provides use with a large speed up(-30%):

# perfchangegroupchangelog on mercurial
! wall 2.010326 comb 2.020000 user 2.000000 sys 0.020000 (best of 5)
! wall 1.382039 comb 1.380000 user 1.370000 sys 0.010000 (best of 8)

# perfchangegroupchangelog on pypy
! wall 5.792589 comb 5.780000 user 5.780000 sys 0.000000 (best of 3)
! wall 3.911158 comb 3.920000 user 3.900000 sys 0.020000 (best of 3)

# perfchangegroupchangelog on mozilla central
! wall 20.683727 comb 20.680000 user 20.630000 sys 0.050000 (best of 3)
! wall 14.190204 comb 14.190000 user 14.150000 sys 0.040000 (best of 3)

Many tests have to be updated because of the change in bundle content. All
theses update have been verified.  Because diffing changelog was not very
valuable, the resulting bundle have similar size (often a bit smaller):

# full bundle of mozilla central
with delta:    1142740533B
without delta: 1142173300B

So this is a win all over the board.
Gregory Szorc - Oct. 14, 2016, 7:01 a.m.
Cool. I was going to author this patch when I got back home!

This patch will result in CPU regression for old clients having to re-deltify. It would be nice to have numbers for that. I'm optimistic it is roughly the same as the server gains and it won't be significant enough to not take the patch. We also don't have a perf* command to measure changegroup application for a single component IIRC. So getting data isn't trivial :/

> On Oct 14, 2016, at 03:08, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
> 
> # HG changeset patch
> # User Pierre-Yves David <pierre-yves.david@ens-lyon.org>
> # Date 1476401471 -7200
> #      Fri Oct 14 01:31:11 2016 +0200
> # Node ID 747c0a1084ef6251a987e469197bad7796756403
> # Parent  e19eb107706e7210c3b359d66f5274911b181566
> # EXP-Topic storedeltachains
> changegroup: skip delta when the underlying revlog do not use them
> 
> Revlog can now be configured to store full snapshot only. This is used on the
> changelog. However, the changegroup packing was still recomputing deltas to be
> sent over the wire.
> 
> We now just reuse the full snapshot directly in this case, skipping delta
> computation. This provides use with a large speed up(-30%):
> 
> # perfchangegroupchangelog on mercurial
> ! wall 2.010326 comb 2.020000 user 2.000000 sys 0.020000 (best of 5)
> ! wall 1.382039 comb 1.380000 user 1.370000 sys 0.010000 (best of 8)
> 
> # perfchangegroupchangelog on pypy
> ! wall 5.792589 comb 5.780000 user 5.780000 sys 0.000000 (best of 3)
> ! wall 3.911158 comb 3.920000 user 3.900000 sys 0.020000 (best of 3)
> 
> # perfchangegroupchangelog on mozilla central
> ! wall 20.683727 comb 20.680000 user 20.630000 sys 0.050000 (best of 3)
> ! wall 14.190204 comb 14.190000 user 14.150000 sys 0.040000 (best of 3)
> 
> Many tests have to be updated because of the change in bundle content. All
> theses update have been verified.  Because diffing changelog was not very
> valuable, the resulting bundle have similar size (often a bit smaller):
> 
> # full bundle of mozilla central
> with delta:    1142740533B
> without delta: 1142173300B
> 
> So this is a win all over the board.
> 
> diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
> --- a/mercurial/changegroup.py
> +++ b/mercurial/changegroup.py
> @@ -818,18 +818,24 @@ class cg2packer(cg1packer):
> 
>     def deltaparent(self, revlog, rev, p1, p2, prev):
>         dp = revlog.deltaparent(rev)
> -        # Avoid sending full revisions when delta parent is null. Pick
> -        # prev in that case. It's tempting to pick p1 in this case, as p1
> -        # will be smaller in the common case. However, computing a delta
> -        # against p1 may require resolving the raw text of p1, which could
> -        # be expensive. The revlog caches should have prev cached, meaning
> -        # less CPU for changegroup generation. There is likely room to add
> -        # a flag and/or config option to control this behavior.
> -        #
> -        # Pick prev when we can't be sure remote has the base revision.
> -        if dp == nullrev or (dp != p1 and dp != p2 and dp != prev):
> +        if dp == nullrev and revlog.storedeltachains:
> +            # Avoid sending full revisions when delta parent is null. Pick prev
> +            # in that case. It's tempting to pick p1 in this case, as p1 will
> +            # be smaller in the common case. However, computing a delta against
> +            # p1 may require resolving the raw text of p1, which could be
> +            # expensive. The revlog caches should have prev cached, meaning
> +            # less CPU for changegroup generation. There is likely room to add
> +            # a flag and/or config option to control this behavior.
>             return prev
> -        return dp
> +        elif dp == nullrev:
> +            # revlog is configure to use full snapshot for a reason,
> +            # stick to full snapshot.
> +            return nullrev
> +        elif dp not in (p1, p2, prev):
> +            # Pick prev when we can't be sure remote has the base revision.
> +            return prev
> +        else:
> +            return dp
> 
>     def builddeltaheader(self, node, p1n, p2n, basenode, linknode, flags):
>         # Do nothing with flags, it is implicitly 0 in cg1 and cg2problem 
> diff --git a/tests/test-acl.t b/tests/test-acl.t
> --- a/tests/test-acl.t
> +++ b/tests/test-acl.t
> @@ -113,7 +113,7 @@ Extension disabled for lack of a hook
>   adding quux/file.py revisions
>   added 3 changesets with 3 changes to 3 files
>   updating the branch cache
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-bundle: 3 parts total
> @@ -178,7 +178,7 @@ Extension disabled for lack of acl.sourc
>   calling hook pretxnchangegroup.acl: hgext.acl.hook
>   acl: changes have source "push" - skipping
>   updating the branch cache
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-bundle: 3 parts total
> @@ -254,7 +254,7 @@ No [acl.allow]/[acl.deny]
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   acl: path access granted: "911600dab2ae"
>   updating the branch cache
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-bundle: 3 parts total
> @@ -325,7 +325,7 @@ Empty [acl.allow]
>   acl: acl.deny not enabled
>   acl: branch access granted: "ef1ea85a6374" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "fred" not allowed on "foo/file.txt" (changeset "ef1ea85a6374")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -394,7 +394,7 @@ fred is allowed inside foo/
>   acl: path access granted: "f9cafe1212c8"
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "fred" not allowed on "quux/file.py" (changeset "911600dab2ae")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -460,7 +460,7 @@ Empty [acl.deny]
>   acl: acl.deny enabled, 0 entries for user barney
>   acl: branch access granted: "ef1ea85a6374" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "barney" not allowed on "foo/file.txt" (changeset "ef1ea85a6374")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -531,7 +531,7 @@ fred is allowed inside foo/, but not foo
>   acl: path access granted: "f9cafe1212c8"
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "fred" not allowed on "quux/file.py" (changeset "911600dab2ae")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -601,7 +601,7 @@ fred is allowed inside foo/, but not foo
>   acl: path access granted: "ef1ea85a6374"
>   acl: branch access granted: "f9cafe1212c8" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "fred" denied on "foo/Bar/file.txt" (changeset "f9cafe1212c8")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -668,7 +668,7 @@ fred is allowed inside foo/, but not foo
>   acl: acl.deny enabled, 0 entries for user barney
>   acl: branch access granted: "ef1ea85a6374" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "barney" not allowed on "foo/file.txt" (changeset "ef1ea85a6374")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -744,7 +744,7 @@ barney is allowed everywhere
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   acl: path access granted: "911600dab2ae"
>   updating the branch cache
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-bundle: 3 parts total
> @@ -826,7 +826,7 @@ wilma can change files with a .txt exten
>   acl: path access granted: "f9cafe1212c8"
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "wilma" not allowed on "quux/file.py" (changeset "911600dab2ae")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -895,7 +895,7 @@ file specified by acl.config does not ex
>   calling hook pretxnchangegroup.acl: hgext.acl.hook
>   acl: checking access for user "barney"
>   error: pretxnchangegroup.acl hook raised an exception: [Errno 2] No such file or directory: '../acl.config'
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -977,7 +977,7 @@ betty is allowed inside foo/ by a acl.co
>   acl: path access granted: "f9cafe1212c8"
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "betty" not allowed on "quux/file.py" (changeset "911600dab2ae")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -1062,7 +1062,7 @@ acl.config can set only [acl.allow]/[acl
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   acl: path access granted: "911600dab2ae"
>   updating the branch cache
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-bundle: 3 parts total
> @@ -1148,7 +1148,7 @@ fred is always allowed
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   acl: path access granted: "911600dab2ae"
>   updating the branch cache
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-bundle: 3 parts total
> @@ -1227,7 +1227,7 @@ no one is allowed inside foo/Bar/
>   acl: path access granted: "ef1ea85a6374"
>   acl: branch access granted: "f9cafe1212c8" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "fred" denied on "foo/Bar/file.txt" (changeset "f9cafe1212c8")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -1307,7 +1307,7 @@ OS-level groups
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   acl: path access granted: "911600dab2ae"
>   updating the branch cache
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-bundle: 3 parts total
> @@ -1388,7 +1388,7 @@ OS-level groups
>   acl: path access granted: "ef1ea85a6374"
>   acl: branch access granted: "f9cafe1212c8" on branch "default"
>   error: pretxnchangegroup.acl hook failed: acl: user "fred" denied on "foo/Bar/file.txt" (changeset "f9cafe1212c8")
> -  bundle2-input-part: total payload size 1606
> +  bundle2-input-part: total payload size 1553
>   bundle2-input-bundle: 3 parts total
>   transaction abort!
>   rollback completed
> @@ -1509,7 +1509,7 @@ No branch acls specified
>   acl: branch access granted: "e8fc755d4d82" on branch "foobar"
>   acl: path access granted: "e8fc755d4d82"
>   updating the branch cache
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
> @@ -1595,7 +1595,7 @@ Branch acl deny test
>   acl: branch access granted: "911600dab2ae" on branch "default"
>   acl: path access granted: "911600dab2ae"
>   error: pretxnchangegroup.acl hook failed: acl: user "astro" denied on branch "foobar" (changeset "e8fc755d4d82")
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-bundle: 4 parts total
>   transaction abort!
>   rollback completed
> @@ -1663,7 +1663,7 @@ Branch acl empty allow test
>   acl: acl.allow not enabled
>   acl: acl.deny not enabled
>   error: pretxnchangegroup.acl hook failed: acl: user "astro" not allowed on branch "default" (changeset "ef1ea85a6374")
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-bundle: 4 parts total
>   transaction abort!
>   rollback completed
> @@ -1733,7 +1733,7 @@ Branch acl allow other
>   acl: acl.allow not enabled
>   acl: acl.deny not enabled
>   error: pretxnchangegroup.acl hook failed: acl: user "astro" not allowed on branch "default" (changeset "ef1ea85a6374")
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-bundle: 4 parts total
>   transaction abort!
>   rollback completed
> @@ -1805,7 +1805,7 @@ Branch acl allow other
>   acl: branch access granted: "e8fc755d4d82" on branch "foobar"
>   acl: path access granted: "e8fc755d4d82"
>   updating the branch cache
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
> @@ -1898,7 +1898,7 @@ push foobar into the remote
>   acl: branch access granted: "e8fc755d4d82" on branch "foobar"
>   acl: path access granted: "e8fc755d4d82"
>   updating the branch cache
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
> @@ -1982,7 +1982,7 @@ Branch acl conflicting deny
>   acl: acl.allow not enabled
>   acl: acl.deny not enabled
>   error: pretxnchangegroup.acl hook failed: acl: user "george" denied on branch "default" (changeset "ef1ea85a6374")
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-bundle: 4 parts total
>   transaction abort!
>   rollback completed
> @@ -2059,7 +2059,7 @@ User 'astro' must not be denied
>   acl: branch access granted: "e8fc755d4d82" on branch "foobar"
>   acl: path access granted: "e8fc755d4d82"
>   updating the branch cache
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
>   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
>   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
> @@ -2137,7 +2137,7 @@ Non-astro users must be denied
>   acl: acl.allow not enabled
>   acl: acl.deny not enabled
>   error: pretxnchangegroup.acl hook failed: acl: user "george" denied on branch "default" (changeset "ef1ea85a6374")
> -  bundle2-input-part: total payload size 2139
> +  bundle2-input-part: total payload size 2068
>   bundle2-input-bundle: 4 parts total
>   transaction abort!
>   rollback completed
> diff --git a/tests/test-clonebundles.t b/tests/test-clonebundles.t
> --- a/tests/test-clonebundles.t
> +++ b/tests/test-clonebundles.t
> @@ -156,34 +156,32 @@ changes, clone bundles produced by new M
> by old clients.
> 
>   $ f --size --hexdump full.hg
> -  full.hg: size=418
> +  full.hg: size=396
>   0000: 48 47 32 30 00 00 00 0e 43 6f 6d 70 72 65 73 73 |HG20....Compress|
>   0010: 69 6f 6e 3d 47 5a 78 9c 63 60 60 d0 e4 76 f6 70 |ion=GZx.c``..v.p|
>   0020: f4 73 77 75 0f f2 0f 0d 60 00 02 46 46 76 26 4e |.swu....`..FFv&N|
>   0030: c6 b2 d4 a2 e2 cc fc 3c 03 a3 bc a4 e4 8c c4 bc |.......<........|
> -  0040: f4 d4 62 23 06 06 e6 65 40 f9 4d c1 2a 31 09 cf |..b#...e@.M.*1..|
> +  0040: f4 d4 62 23 06 06 e6 19 40 f9 4d c1 2a 31 09 cf |..b#....@.M.*1..|
>   0050: 9a 3a 52 04 b7 fc db f0 95 e5 a4 f4 97 17 b2 c9 |.:R.............|
>   0060: 0c 14 00 02 e6 d9 99 25 1a a7 a4 99 a4 a4 1a 5b |.......%.......[|
>   0070: 58 a4 19 27 9b a4 59 a4 1a 59 a4 99 a4 59 26 5a |X..'..Y..Y...Y&Z|
>   0080: 18 9a 18 59 5a 26 1a 27 27 25 99 a6 99 1a 70 95 |...YZ&.''%....p.|
>   0090: a4 16 97 70 19 28 18 70 a5 e5 e7 73 71 25 a6 a4 |...p.(.p...sq%..|
> -  00a0: 28 00 19 40 13 0e ac fa df ab ff 7b 3f fb 92 dc |(..@.......{?...|
> -  00b0: 8b 1f 62 bb 9e b7 d7 d9 87 3d 5a 44 ac 2f b0 a9 |..b......=ZD./..|
> -  00c0: c3 66 1e 54 b9 26 08 a7 1a 1b 1a a7 25 1b 9a 1b |.f.T.&......%...|
> -  00d0: 99 19 9a 5a 18 9b a6 18 19 00 dd 67 61 61 98 06 |...Z.......gaa..|
> -  00e0: f4 80 49 4a 8a 65 52 92 41 9a 81 81 a5 11 17 50 |..IJ.eR.A......P|
> -  00f0: 31 30 58 19 cc 80 98 25 29 b1 08 c4 37 07 79 19 |10X....%)...7.y.|
> -  0100: 88 d9 41 ee 07 8a 41 cd 5d 98 65 fb e5 9e 45 bf |..A...A.].e...E.|
> -  0110: 8d 7f 9f c6 97 9f 2b 44 34 67 d9 ec 8e 0f a0 61 |......+D4g.....a|
> -  0120: a8 eb 82 82 2e c9 c2 20 25 d5 34 c5 d0 d8 c2 dc |....... %.4.....|
> -  0130: d4 c2 d4 c4 30 d9 34 cd c0 d4 c8 cc 34 31 c5 d0 |....0.4.....41..|
> -  0140: c4 24 31 c9 32 2d d1 c2 2c c5 30 25 09 e4 ee 85 |.$1.2-..,.0%....|
> -  0150: 8f 85 ff 88 ab 89 36 c7 2a c4 47 34 fe f8 ec 7b |......6.*.G4...{|
> -  0160: 73 37 3f c3 24 62 1d 8d 4d 1d 9e 40 06 3b 10 14 |s7?.$b..M..@.;..|
> -  0170: 36 a4 38 10 04 d8 21 01 9a b1 83 f7 e9 45 8b d2 |6.8...!......E..|
> -  0180: 56 c7 a3 1f 82 52 d7 8a 78 ed fc d5 76 f1 36 25 |V....R..x...v.6%|
> -  0190: 81 89 c7 ad ec 90 34 48 75 2b 89 49 bf 00 cf 72 |......4Hu+.I...r|
> -  01a0: f4 7f                                           |..|
> +  00a0: 28 00 19 20 17 af fa df ab ff 7b 3f fb 92 dc 8b |(.. ......{?....|
> +  00b0: 1f 62 bb 9e b7 d7 d9 87 3d 5a 44 89 2f b0 99 87 |.b......=ZD./...|
> +  00c0: ec e2 54 63 43 e3 b4 64 43 73 23 33 43 53 0b 63 |..TcC..dCs#3CS.c|
> +  00d0: d3 14 23 03 a0 fb 2c 2c 0c d3 80 1e 30 49 49 b1 |..#...,,....0II.|
> +  00e0: 4c 4a 32 48 33 30 b0 34 42 b8 38 29 b1 08 e2 62 |LJ2H30.4B.8)...b|
> +  00f0: 20 03 6a ca c2 2c db 2f f7 2c fa 6d fc fb 34 be | .j..,./.,.m..4.|
> +  0100: fc 5c 21 a2 39 cb 66 77 7c 00 0d c3 59 17 14 58 |.\!.9.fw|...Y..X|
> +  0110: 49 16 06 29 a9 a6 29 86 c6 16 e6 a6 16 a6 26 86 |I..)..).......&.|
> +  0120: c9 a6 69 06 a6 46 66 a6 89 29 86 26 26 89 49 96 |..i..Ff..).&&.I.|
> +  0130: 69 89 16 66 29 86 29 49 5c 20 07 3e 16 fe 23 ae |i..f).)I\ .>..#.|
> +  0140: 26 da 1c ab 10 1f d1 f8 e3 b3 ef cd dd fc 0c 93 |&...............|
> +  0150: 88 75 34 36 75 04 82 55 17 14 36 a4 38 10 04 d8 |.u46u..U..6.8...|
> +  0160: 21 01 9a b1 83 f7 e9 45 8b d2 56 c7 a3 1f 82 52 |!......E..V....R|
> +  0170: d7 8a 78 ed fc d5 76 f1 36 25 81 89 c7 ad ec 90 |..x...v.6%......|
> +  0180: 54 47 75 2b 89 49 b1 00 d2 8a eb 92             |TGu+.I......|
> 
>   $ echo "http://localhost:$HGPORT1/full.hg" > server/.hg/clonebundles.manifest
>   $ hg clone -U http://localhost:$HGPORT full-bundle
> diff --git a/tests/test-commit-amend.t b/tests/test-commit-amend.t
> --- a/tests/test-commit-amend.t
> +++ b/tests/test-commit-amend.t
> @@ -120,13 +120,13 @@ No changes, just a different message:
>   stripping amended changeset 74609c7f506e
>   1 changesets found
>   uncompressed size of bundle content:
> -       270 (changelog)
> +       254 (changelog)
>        163 (manifests)
>        129  a
>   saved backup bundle to $TESTTMP/.hg/strip-backup/74609c7f506e-1bfde511-amend-backup.hg (glob)
>   1 changesets found
>   uncompressed size of bundle content:
> -       266 (changelog)
> +       250 (changelog)
>        163 (manifests)
>        129  a
>   adding branch
> @@ -264,13 +264,13 @@ then, test editing custom commit message
>   stripping amended changeset 5f357c7560ab
>   1 changesets found
>   uncompressed size of bundle content:
> -       258 (changelog)
> +       249 (changelog)
>        163 (manifests)
>        131  a
>   saved backup bundle to $TESTTMP/.hg/strip-backup/5f357c7560ab-e7c84ade-amend-backup.hg (glob)
>   1 changesets found
>   uncompressed size of bundle content:
> -       266 (changelog)
> +       257 (changelog)
>        163 (manifests)
>        131  a
>   adding branch
> @@ -307,13 +307,13 @@ Same, but with changes in working dir (d
>   stripping amended changeset 7ab3bf440b54
>   2 changesets found
>   uncompressed size of bundle content:
> -       490 (changelog)
> +       464 (changelog)
>        322 (manifests)
>        249  a
>   saved backup bundle to $TESTTMP/.hg/strip-backup/7ab3bf440b54-8e3b5088-amend-backup.hg (glob)
>   1 changesets found
>   uncompressed size of bundle content:
> -       266 (changelog)
> +       257 (changelog)
>        163 (manifests)
>        133  a
>   adding branch
> diff --git a/tests/test-debugbundle.t b/tests/test-debugbundle.t
> --- a/tests/test-debugbundle.t
> +++ b/tests/test-debugbundle.t
> @@ -16,7 +16,7 @@ Create a test repository:
>   $ hg bundle --base 0 --rev tip bundle2.hg -v --type none-v2
>   2 changesets found
>   uncompressed size of bundle content:
> -       372 (changelog)
> +       344 (changelog)
>        322 (manifests)
>        113  b
>        113  c
> @@ -60,8 +60,8 @@ Verbose output:
>       format: id, p1, p2, cset, delta base, len(delta)
> 
>       changelog
> -      0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 80
> -      991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 80
> +      0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 66
> +      991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 0000000000000000000000000000000000000000 66
> 
>       manifest
>       686dbf0aeca417636fa26a9121c681eabbb15a20 8515d4bfda768e04af4c13a69a72e28c7effbea7 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 8515d4bfda768e04af4c13a69a72e28c7effbea7 55
> diff --git a/tests/test-largefiles.t b/tests/test-largefiles.t
> --- a/tests/test-largefiles.t
> +++ b/tests/test-largefiles.t
> @@ -1100,7 +1100,7 @@ redo pull with --lfrev and check it pull
>   all local heads known remotely
>   6 changesets found
>   uncompressed size of bundle content:
> -      1333 (changelog)
> +      1389 (changelog)
>       1599 (manifests)
>        254  .hglf/large1
>        564  .hglf/large3
> diff --git a/tests/test-obsolete-changeset-exchange.t b/tests/test-obsolete-changeset-exchange.t
> --- a/tests/test-obsolete-changeset-exchange.t
> +++ b/tests/test-obsolete-changeset-exchange.t
> @@ -145,7 +145,7 @@ client only pulls down 1 changeset
>   adding foo revisions
>   added 1 changesets with 1 changes to 1 files (+1 heads)
>   updating the branch cache
> -  bundle2-input-part: total payload size 474
> +  bundle2-input-part: total payload size 476
>   bundle2-input-part: "listkeys" (params: 1 mandatory) supported
>   bundle2-input-part: total payload size 58
>   bundle2-input-part: "listkeys" (params: 1 mandatory) supported
> diff --git a/tests/test-patchbomb.t b/tests/test-patchbomb.t
> --- a/tests/test-patchbomb.t
> +++ b/tests/test-patchbomb.t
> @@ -352,14 +352,14 @@ test bundle and description:
>   Content-Disposition: attachment; filename="bundle.hg"
>   Content-Transfer-Encoding: base64
> 
> -  SEcyMAAAAA5Db21wcmVzc2lvbj1CWkJaaDkxQVkmU1kIqE7KAAAKf//7vFYQWD1/4H7R09C/470I
> -  Ak0E4peoSIYIgQCgGUQOcLABGY2hqoTTCYaBqaYAAACaaMAATIwAA1MIYDaQaqn6p+jRop+oJkA2
> -  oNqD1PU0PUBoxqaMmjMUepoBoDT1GmQNBKmlTT1GTCNMEAYTQ0NNDI0BoMQHpAZAA8o2pkyNJHfX
> -  RRbXoyxKRUlAg41B3lpmMOnr77dEpFKAvEUGEkWuC4wioiMjC2Y2a84EXhsNCFIrbXUGId07PJnS
> -  ELAOIpL/gE8R8CUeXuw2NKMtkFoLPkcTSomXtgHSg1IKaCNlWwVU3CpmMYqh5gkFYJKOD4UhVVQ6
> -  SiF1DpE8ghWvF1ih+fYgagfYHI96w/QsrRATpYiP7VRbINFrQy2c21mZ7M4pXXrPBypoXAIhtum7
> -  aKDJCpUqMDF5dfiDChMfgH9nQ4B60Uvgb4AK9dsbSYc+O3tEyNq9g9gZeA5Je2T82GzjC4DbY4F2
> -  0kdrTBwslErFshCgDzeEBwICg13oQaQawQA1WWd3F3JFOFCQCKhOyg==
> +  SEcyMAAAAA5Db21wcmVzc2lvbj1CWkJaaDkxQVkmU1nYvy2xAAAJf//7vFYQXD1/4H7R09C/470I
> +  Ak0E4pe4SIIIgQSgGEQOcLAA5VBKqeppMxTI0YjQNBgQMQDI0GgMhtR6I0GI2p6I0yeSEVT9MiYn
> +  qjCYQwCBtARptARgBNDEwAGiDCMA40NGjQaNA0AAAAADIAAAA0BkABktCk6qObVxZ2A/33KHibLr
> +  UQ4BwkgcPcmuCUAQZCztIWgR1SpBS6IqbIij4UFwhnnMkElcFTqoucIWbsBPK3l+6c+xYaVBWsJo
> +  aT0OV/YAOvLrziifDQMJOMIaaYce9agtI2EwQBAq089UiRU+evFHSLRBT7Wa/D/YBaUtU5ezvtr3
> +  6yrIS4Iyp9VWESdWPEi6VjRjdcEY4HvbmDIVEAEVJIUrHNIBx/MmnBBRkw8tSlCQ8ABZxf5ejgBI
> +  pP5TSQPLVMYbq1qbBPmWN0LYVlAvRbP4X512kDQZ9y4TQbvoZmhe+54sRsEJ8GW3hMJjERh0NNlg
> +  aB+3Cw/4u5IpwoSGxfltiA==
>   --===============*==-- (glob)
> 
> with a specific bundle type
> diff --git a/tests/test-phases-exchange.t b/tests/test-phases-exchange.t
> --- a/tests/test-phases-exchange.t
> +++ b/tests/test-phases-exchange.t
> @@ -765,7 +765,7 @@ Bare push with next changeset and common
>   searching for changes
>   1 changesets found
>   uncompressed size of bundle content:
> -       192 (changelog)
> +       178 (changelog)
>        165 (manifests)
>        131  a-H
>   adding changesets
> diff --git a/tests/test-push-warn.t b/tests/test-push-warn.t
> --- a/tests/test-push-warn.t
> +++ b/tests/test-push-warn.t
> @@ -150,7 +150,7 @@ Specifying a revset that evaluates to nu
>   searching for changes
>   2 changesets found
>   uncompressed size of bundle content:
> -       348 (changelog)
> +       352 (changelog)
>        326 (manifests)
>        253  foo
>   adding changesets
> diff --git a/tests/test-rebase-conflicts.t b/tests/test-rebase-conflicts.t
> --- a/tests/test-rebase-conflicts.t
> +++ b/tests/test-rebase-conflicts.t
> @@ -299,7 +299,7 @@ Check that the right ancestors is used w
>   adding file changes
>   adding f1.txt revisions
>   added 2 changesets with 2 changes to 1 files
> -  bundle2-input-part: total payload size 1713
> +  bundle2-input-part: total payload size 1686
>   bundle2-input-bundle: 0 parts total
>   invalid branchheads cache (served): tip differs
>   history modification detected - truncating revision branch cache to revision 9
> diff --git a/tests/test-rebase-mq-skip.t b/tests/test-rebase-mq-skip.t
> --- a/tests/test-rebase-mq-skip.t
> +++ b/tests/test-rebase-mq-skip.t
> @@ -70,14 +70,14 @@ already has one local mq patch
>   $TESTTMP/a/.hg/patches/p0.patch (glob)
>   2 changesets found
>   uncompressed size of bundle content:
> -       384 (changelog)
> +       348 (changelog)
>        324 (manifests)
>        129  p0
>        129  p1
>   saved backup bundle to $TESTTMP/a/.hg/strip-backup/13a46ce44f60-5da6ecfb-backup.hg (glob)
>   2 changesets found
>   uncompressed size of bundle content:
> -       439 (changelog)
> +       403 (changelog)
>        324 (manifests)
>        129  p0
>        129  p1
> diff --git a/tests/test-rebase-newancestor.t b/tests/test-rebase-newancestor.t
> --- a/tests/test-rebase-newancestor.t
> +++ b/tests/test-rebase-newancestor.t
> @@ -304,13 +304,13 @@ rebase of merge of ancestors
>   rebase merging completed
>   1 changesets found
>   uncompressed size of bundle content:
> -       213 (changelog)
> +       199 (changelog)
>        216 (manifests)
>        182  other
>   saved backup bundle to $TESTTMP/parentorder/.hg/strip-backup/4c5f12f25ebe-f46990e5-backup.hg (glob)
>   1 changesets found
>   uncompressed size of bundle content:
> -       272 (changelog)
> +       254 (changelog)
>        167 (manifests)
>        182  other
>   adding branch
Pierre-Yves David - Oct. 16, 2016, 12:02 p.m.
On 10/14/2016 09:01 AM, Gregory Szorc wrote:
> Cool. I was going to author this patch when I got back home!
>
> This patch will result in CPU regression for old clients having to re-deltify. It would be nice to have numbers for that. I'm optimistic it is roughly the same as the server gains and it won't be significant enough to not take the patch. We also don't have a perf* command to measure changegroup application for a single component IIRC. So getting data isn't trivial :/

As you just said, the redeltifying are probably around the same cost 
than the one we save with this patch (6 seconds for a full Mozilla 
clone). The extra cost seems reasonable to me in that case.

Cheers,
Yuya Nishihara - Oct. 18, 2016, 12:40 p.m.
On Fri, 14 Oct 2016 03:08:06 +0200, Pierre-Yves David wrote:
> # HG changeset patch
> # User Pierre-Yves David <pierre-yves.david@ens-lyon.org>
> # Date 1476401471 -7200
> #      Fri Oct 14 01:31:11 2016 +0200
> # Node ID 747c0a1084ef6251a987e469197bad7796756403
> # Parent  e19eb107706e7210c3b359d66f5274911b181566
> # EXP-Topic storedeltachains
> changegroup: skip delta when the underlying revlog do not use them

Seems fine per the discussion on this thread. Queued, thanks.
Augie Fackler - Oct. 18, 2016, 1:11 p.m.
On Sun, Oct 16, 2016 at 02:02:23PM +0200, Pierre-Yves David wrote:
>
>
> On 10/14/2016 09:01 AM, Gregory Szorc wrote:
> >Cool. I was going to author this patch when I got back home!
> >
> >This patch will result in CPU regression for old clients having to re-deltify. It would be nice to have numbers for that. I'm optimistic it is roughly the same as the server gains and it won't be significant enough to not take the patch. We also don't have a perf* command to measure changegroup application for a single component IIRC. So getting data isn't trivial :/
>
> As you just said, the redeltifying are probably around the same cost than
> the one we save with this patch (6 seconds for a full Mozilla clone). The
> extra cost seems reasonable to me in that case.

Queued these, thanks.

>
> Cheers,
>
> --
> Pierre-Yves David
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Patch

diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -818,18 +818,24 @@  class cg2packer(cg1packer):
 
     def deltaparent(self, revlog, rev, p1, p2, prev):
         dp = revlog.deltaparent(rev)
-        # Avoid sending full revisions when delta parent is null. Pick
-        # prev in that case. It's tempting to pick p1 in this case, as p1
-        # will be smaller in the common case. However, computing a delta
-        # against p1 may require resolving the raw text of p1, which could
-        # be expensive. The revlog caches should have prev cached, meaning
-        # less CPU for changegroup generation. There is likely room to add
-        # a flag and/or config option to control this behavior.
-        #
-        # Pick prev when we can't be sure remote has the base revision.
-        if dp == nullrev or (dp != p1 and dp != p2 and dp != prev):
+        if dp == nullrev and revlog.storedeltachains:
+            # Avoid sending full revisions when delta parent is null. Pick prev
+            # in that case. It's tempting to pick p1 in this case, as p1 will
+            # be smaller in the common case. However, computing a delta against
+            # p1 may require resolving the raw text of p1, which could be
+            # expensive. The revlog caches should have prev cached, meaning
+            # less CPU for changegroup generation. There is likely room to add
+            # a flag and/or config option to control this behavior.
             return prev
-        return dp
+        elif dp == nullrev:
+            # revlog is configure to use full snapshot for a reason,
+            # stick to full snapshot.
+            return nullrev
+        elif dp not in (p1, p2, prev):
+            # Pick prev when we can't be sure remote has the base revision.
+            return prev
+        else:
+            return dp
 
     def builddeltaheader(self, node, p1n, p2n, basenode, linknode, flags):
         # Do nothing with flags, it is implicitly 0 in cg1 and cg2
diff --git a/tests/test-acl.t b/tests/test-acl.t
--- a/tests/test-acl.t
+++ b/tests/test-acl.t
@@ -113,7 +113,7 @@  Extension disabled for lack of a hook
   adding quux/file.py revisions
   added 3 changesets with 3 changes to 3 files
   updating the branch cache
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-bundle: 3 parts total
@@ -178,7 +178,7 @@  Extension disabled for lack of acl.sourc
   calling hook pretxnchangegroup.acl: hgext.acl.hook
   acl: changes have source "push" - skipping
   updating the branch cache
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-bundle: 3 parts total
@@ -254,7 +254,7 @@  No [acl.allow]/[acl.deny]
   acl: branch access granted: "911600dab2ae" on branch "default"
   acl: path access granted: "911600dab2ae"
   updating the branch cache
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-bundle: 3 parts total
@@ -325,7 +325,7 @@  Empty [acl.allow]
   acl: acl.deny not enabled
   acl: branch access granted: "ef1ea85a6374" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "fred" not allowed on "foo/file.txt" (changeset "ef1ea85a6374")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -394,7 +394,7 @@  fred is allowed inside foo/
   acl: path access granted: "f9cafe1212c8"
   acl: branch access granted: "911600dab2ae" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "fred" not allowed on "quux/file.py" (changeset "911600dab2ae")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -460,7 +460,7 @@  Empty [acl.deny]
   acl: acl.deny enabled, 0 entries for user barney
   acl: branch access granted: "ef1ea85a6374" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "barney" not allowed on "foo/file.txt" (changeset "ef1ea85a6374")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -531,7 +531,7 @@  fred is allowed inside foo/, but not foo
   acl: path access granted: "f9cafe1212c8"
   acl: branch access granted: "911600dab2ae" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "fred" not allowed on "quux/file.py" (changeset "911600dab2ae")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -601,7 +601,7 @@  fred is allowed inside foo/, but not foo
   acl: path access granted: "ef1ea85a6374"
   acl: branch access granted: "f9cafe1212c8" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "fred" denied on "foo/Bar/file.txt" (changeset "f9cafe1212c8")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -668,7 +668,7 @@  fred is allowed inside foo/, but not foo
   acl: acl.deny enabled, 0 entries for user barney
   acl: branch access granted: "ef1ea85a6374" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "barney" not allowed on "foo/file.txt" (changeset "ef1ea85a6374")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -744,7 +744,7 @@  barney is allowed everywhere
   acl: branch access granted: "911600dab2ae" on branch "default"
   acl: path access granted: "911600dab2ae"
   updating the branch cache
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-bundle: 3 parts total
@@ -826,7 +826,7 @@  wilma can change files with a .txt exten
   acl: path access granted: "f9cafe1212c8"
   acl: branch access granted: "911600dab2ae" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "wilma" not allowed on "quux/file.py" (changeset "911600dab2ae")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -895,7 +895,7 @@  file specified by acl.config does not ex
   calling hook pretxnchangegroup.acl: hgext.acl.hook
   acl: checking access for user "barney"
   error: pretxnchangegroup.acl hook raised an exception: [Errno 2] No such file or directory: '../acl.config'
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -977,7 +977,7 @@  betty is allowed inside foo/ by a acl.co
   acl: path access granted: "f9cafe1212c8"
   acl: branch access granted: "911600dab2ae" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "betty" not allowed on "quux/file.py" (changeset "911600dab2ae")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -1062,7 +1062,7 @@  acl.config can set only [acl.allow]/[acl
   acl: branch access granted: "911600dab2ae" on branch "default"
   acl: path access granted: "911600dab2ae"
   updating the branch cache
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-bundle: 3 parts total
@@ -1148,7 +1148,7 @@  fred is always allowed
   acl: branch access granted: "911600dab2ae" on branch "default"
   acl: path access granted: "911600dab2ae"
   updating the branch cache
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-bundle: 3 parts total
@@ -1227,7 +1227,7 @@  no one is allowed inside foo/Bar/
   acl: path access granted: "ef1ea85a6374"
   acl: branch access granted: "f9cafe1212c8" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "fred" denied on "foo/Bar/file.txt" (changeset "f9cafe1212c8")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -1307,7 +1307,7 @@  OS-level groups
   acl: branch access granted: "911600dab2ae" on branch "default"
   acl: path access granted: "911600dab2ae"
   updating the branch cache
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-bundle: 3 parts total
@@ -1388,7 +1388,7 @@  OS-level groups
   acl: path access granted: "ef1ea85a6374"
   acl: branch access granted: "f9cafe1212c8" on branch "default"
   error: pretxnchangegroup.acl hook failed: acl: user "fred" denied on "foo/Bar/file.txt" (changeset "f9cafe1212c8")
-  bundle2-input-part: total payload size 1606
+  bundle2-input-part: total payload size 1553
   bundle2-input-bundle: 3 parts total
   transaction abort!
   rollback completed
@@ -1509,7 +1509,7 @@  No branch acls specified
   acl: branch access granted: "e8fc755d4d82" on branch "foobar"
   acl: path access granted: "e8fc755d4d82"
   updating the branch cache
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
@@ -1595,7 +1595,7 @@  Branch acl deny test
   acl: branch access granted: "911600dab2ae" on branch "default"
   acl: path access granted: "911600dab2ae"
   error: pretxnchangegroup.acl hook failed: acl: user "astro" denied on branch "foobar" (changeset "e8fc755d4d82")
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-bundle: 4 parts total
   transaction abort!
   rollback completed
@@ -1663,7 +1663,7 @@  Branch acl empty allow test
   acl: acl.allow not enabled
   acl: acl.deny not enabled
   error: pretxnchangegroup.acl hook failed: acl: user "astro" not allowed on branch "default" (changeset "ef1ea85a6374")
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-bundle: 4 parts total
   transaction abort!
   rollback completed
@@ -1733,7 +1733,7 @@  Branch acl allow other
   acl: acl.allow not enabled
   acl: acl.deny not enabled
   error: pretxnchangegroup.acl hook failed: acl: user "astro" not allowed on branch "default" (changeset "ef1ea85a6374")
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-bundle: 4 parts total
   transaction abort!
   rollback completed
@@ -1805,7 +1805,7 @@  Branch acl allow other
   acl: branch access granted: "e8fc755d4d82" on branch "foobar"
   acl: path access granted: "e8fc755d4d82"
   updating the branch cache
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
@@ -1898,7 +1898,7 @@  push foobar into the remote
   acl: branch access granted: "e8fc755d4d82" on branch "foobar"
   acl: path access granted: "e8fc755d4d82"
   updating the branch cache
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
@@ -1982,7 +1982,7 @@  Branch acl conflicting deny
   acl: acl.allow not enabled
   acl: acl.deny not enabled
   error: pretxnchangegroup.acl hook failed: acl: user "george" denied on branch "default" (changeset "ef1ea85a6374")
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-bundle: 4 parts total
   transaction abort!
   rollback completed
@@ -2059,7 +2059,7 @@  User 'astro' must not be denied
   acl: branch access granted: "e8fc755d4d82" on branch "foobar"
   acl: path access granted: "e8fc755d4d82"
   updating the branch cache
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
   pushing key for "phases:911600dab2ae7a9baff75958b84fe606851ce955"
   bundle2-input-part: "pushkey" (params: 4 mandatory) supported
@@ -2137,7 +2137,7 @@  Non-astro users must be denied
   acl: acl.allow not enabled
   acl: acl.deny not enabled
   error: pretxnchangegroup.acl hook failed: acl: user "george" denied on branch "default" (changeset "ef1ea85a6374")
-  bundle2-input-part: total payload size 2139
+  bundle2-input-part: total payload size 2068
   bundle2-input-bundle: 4 parts total
   transaction abort!
   rollback completed
diff --git a/tests/test-clonebundles.t b/tests/test-clonebundles.t
--- a/tests/test-clonebundles.t
+++ b/tests/test-clonebundles.t
@@ -156,34 +156,32 @@  changes, clone bundles produced by new M
 by old clients.
 
   $ f --size --hexdump full.hg
-  full.hg: size=418
+  full.hg: size=396
   0000: 48 47 32 30 00 00 00 0e 43 6f 6d 70 72 65 73 73 |HG20....Compress|
   0010: 69 6f 6e 3d 47 5a 78 9c 63 60 60 d0 e4 76 f6 70 |ion=GZx.c``..v.p|
   0020: f4 73 77 75 0f f2 0f 0d 60 00 02 46 46 76 26 4e |.swu....`..FFv&N|
   0030: c6 b2 d4 a2 e2 cc fc 3c 03 a3 bc a4 e4 8c c4 bc |.......<........|
-  0040: f4 d4 62 23 06 06 e6 65 40 f9 4d c1 2a 31 09 cf |..b#...e@.M.*1..|
+  0040: f4 d4 62 23 06 06 e6 19 40 f9 4d c1 2a 31 09 cf |..b#....@.M.*1..|
   0050: 9a 3a 52 04 b7 fc db f0 95 e5 a4 f4 97 17 b2 c9 |.:R.............|
   0060: 0c 14 00 02 e6 d9 99 25 1a a7 a4 99 a4 a4 1a 5b |.......%.......[|
   0070: 58 a4 19 27 9b a4 59 a4 1a 59 a4 99 a4 59 26 5a |X..'..Y..Y...Y&Z|
   0080: 18 9a 18 59 5a 26 1a 27 27 25 99 a6 99 1a 70 95 |...YZ&.''%....p.|
   0090: a4 16 97 70 19 28 18 70 a5 e5 e7 73 71 25 a6 a4 |...p.(.p...sq%..|
-  00a0: 28 00 19 40 13 0e ac fa df ab ff 7b 3f fb 92 dc |(..@.......{?...|
-  00b0: 8b 1f 62 bb 9e b7 d7 d9 87 3d 5a 44 ac 2f b0 a9 |..b......=ZD./..|
-  00c0: c3 66 1e 54 b9 26 08 a7 1a 1b 1a a7 25 1b 9a 1b |.f.T.&......%...|
-  00d0: 99 19 9a 5a 18 9b a6 18 19 00 dd 67 61 61 98 06 |...Z.......gaa..|
-  00e0: f4 80 49 4a 8a 65 52 92 41 9a 81 81 a5 11 17 50 |..IJ.eR.A......P|
-  00f0: 31 30 58 19 cc 80 98 25 29 b1 08 c4 37 07 79 19 |10X....%)...7.y.|
-  0100: 88 d9 41 ee 07 8a 41 cd 5d 98 65 fb e5 9e 45 bf |..A...A.].e...E.|
-  0110: 8d 7f 9f c6 97 9f 2b 44 34 67 d9 ec 8e 0f a0 61 |......+D4g.....a|
-  0120: a8 eb 82 82 2e c9 c2 20 25 d5 34 c5 d0 d8 c2 dc |....... %.4.....|
-  0130: d4 c2 d4 c4 30 d9 34 cd c0 d4 c8 cc 34 31 c5 d0 |....0.4.....41..|
-  0140: c4 24 31 c9 32 2d d1 c2 2c c5 30 25 09 e4 ee 85 |.$1.2-..,.0%....|
-  0150: 8f 85 ff 88 ab 89 36 c7 2a c4 47 34 fe f8 ec 7b |......6.*.G4...{|
-  0160: 73 37 3f c3 24 62 1d 8d 4d 1d 9e 40 06 3b 10 14 |s7?.$b..M..@.;..|
-  0170: 36 a4 38 10 04 d8 21 01 9a b1 83 f7 e9 45 8b d2 |6.8...!......E..|
-  0180: 56 c7 a3 1f 82 52 d7 8a 78 ed fc d5 76 f1 36 25 |V....R..x...v.6%|
-  0190: 81 89 c7 ad ec 90 34 48 75 2b 89 49 bf 00 cf 72 |......4Hu+.I...r|
-  01a0: f4 7f                                           |..|
+  00a0: 28 00 19 20 17 af fa df ab ff 7b 3f fb 92 dc 8b |(.. ......{?....|
+  00b0: 1f 62 bb 9e b7 d7 d9 87 3d 5a 44 89 2f b0 99 87 |.b......=ZD./...|
+  00c0: ec e2 54 63 43 e3 b4 64 43 73 23 33 43 53 0b 63 |..TcC..dCs#3CS.c|
+  00d0: d3 14 23 03 a0 fb 2c 2c 0c d3 80 1e 30 49 49 b1 |..#...,,....0II.|
+  00e0: 4c 4a 32 48 33 30 b0 34 42 b8 38 29 b1 08 e2 62 |LJ2H30.4B.8)...b|
+  00f0: 20 03 6a ca c2 2c db 2f f7 2c fa 6d fc fb 34 be | .j..,./.,.m..4.|
+  0100: fc 5c 21 a2 39 cb 66 77 7c 00 0d c3 59 17 14 58 |.\!.9.fw|...Y..X|
+  0110: 49 16 06 29 a9 a6 29 86 c6 16 e6 a6 16 a6 26 86 |I..)..).......&.|
+  0120: c9 a6 69 06 a6 46 66 a6 89 29 86 26 26 89 49 96 |..i..Ff..).&&.I.|
+  0130: 69 89 16 66 29 86 29 49 5c 20 07 3e 16 fe 23 ae |i..f).)I\ .>..#.|
+  0140: 26 da 1c ab 10 1f d1 f8 e3 b3 ef cd dd fc 0c 93 |&...............|
+  0150: 88 75 34 36 75 04 82 55 17 14 36 a4 38 10 04 d8 |.u46u..U..6.8...|
+  0160: 21 01 9a b1 83 f7 e9 45 8b d2 56 c7 a3 1f 82 52 |!......E..V....R|
+  0170: d7 8a 78 ed fc d5 76 f1 36 25 81 89 c7 ad ec 90 |..x...v.6%......|
+  0180: 54 47 75 2b 89 49 b1 00 d2 8a eb 92             |TGu+.I......|
 
   $ echo "http://localhost:$HGPORT1/full.hg" > server/.hg/clonebundles.manifest
   $ hg clone -U http://localhost:$HGPORT full-bundle
diff --git a/tests/test-commit-amend.t b/tests/test-commit-amend.t
--- a/tests/test-commit-amend.t
+++ b/tests/test-commit-amend.t
@@ -120,13 +120,13 @@  No changes, just a different message:
   stripping amended changeset 74609c7f506e
   1 changesets found
   uncompressed size of bundle content:
-       270 (changelog)
+       254 (changelog)
        163 (manifests)
        129  a
   saved backup bundle to $TESTTMP/.hg/strip-backup/74609c7f506e-1bfde511-amend-backup.hg (glob)
   1 changesets found
   uncompressed size of bundle content:
-       266 (changelog)
+       250 (changelog)
        163 (manifests)
        129  a
   adding branch
@@ -264,13 +264,13 @@  then, test editing custom commit message
   stripping amended changeset 5f357c7560ab
   1 changesets found
   uncompressed size of bundle content:
-       258 (changelog)
+       249 (changelog)
        163 (manifests)
        131  a
   saved backup bundle to $TESTTMP/.hg/strip-backup/5f357c7560ab-e7c84ade-amend-backup.hg (glob)
   1 changesets found
   uncompressed size of bundle content:
-       266 (changelog)
+       257 (changelog)
        163 (manifests)
        131  a
   adding branch
@@ -307,13 +307,13 @@  Same, but with changes in working dir (d
   stripping amended changeset 7ab3bf440b54
   2 changesets found
   uncompressed size of bundle content:
-       490 (changelog)
+       464 (changelog)
        322 (manifests)
        249  a
   saved backup bundle to $TESTTMP/.hg/strip-backup/7ab3bf440b54-8e3b5088-amend-backup.hg (glob)
   1 changesets found
   uncompressed size of bundle content:
-       266 (changelog)
+       257 (changelog)
        163 (manifests)
        133  a
   adding branch
diff --git a/tests/test-debugbundle.t b/tests/test-debugbundle.t
--- a/tests/test-debugbundle.t
+++ b/tests/test-debugbundle.t
@@ -16,7 +16,7 @@  Create a test repository:
   $ hg bundle --base 0 --rev tip bundle2.hg -v --type none-v2
   2 changesets found
   uncompressed size of bundle content:
-       372 (changelog)
+       344 (changelog)
        322 (manifests)
        113  b
        113  c
@@ -60,8 +60,8 @@  Verbose output:
       format: id, p1, p2, cset, delta base, len(delta)
   
       changelog
-      0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 80
-      991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 80
+      0e067c57feba1a5694ca4844f05588bb1bf82342 3903775176ed42b1458a6281db4a0ccf4d9f287a 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 66
+      991a3460af53952d10ec8a295d3d2cc2e5fa9690 0e067c57feba1a5694ca4844f05588bb1bf82342 0000000000000000000000000000000000000000 991a3460af53952d10ec8a295d3d2cc2e5fa9690 0000000000000000000000000000000000000000 66
   
       manifest
       686dbf0aeca417636fa26a9121c681eabbb15a20 8515d4bfda768e04af4c13a69a72e28c7effbea7 0000000000000000000000000000000000000000 0e067c57feba1a5694ca4844f05588bb1bf82342 8515d4bfda768e04af4c13a69a72e28c7effbea7 55
diff --git a/tests/test-largefiles.t b/tests/test-largefiles.t
--- a/tests/test-largefiles.t
+++ b/tests/test-largefiles.t
@@ -1100,7 +1100,7 @@  redo pull with --lfrev and check it pull
   all local heads known remotely
   6 changesets found
   uncompressed size of bundle content:
-      1333 (changelog)
+      1389 (changelog)
       1599 (manifests)
        254  .hglf/large1
        564  .hglf/large3
diff --git a/tests/test-obsolete-changeset-exchange.t b/tests/test-obsolete-changeset-exchange.t
--- a/tests/test-obsolete-changeset-exchange.t
+++ b/tests/test-obsolete-changeset-exchange.t
@@ -145,7 +145,7 @@  client only pulls down 1 changeset
   adding foo revisions
   added 1 changesets with 1 changes to 1 files (+1 heads)
   updating the branch cache
-  bundle2-input-part: total payload size 474
+  bundle2-input-part: total payload size 476
   bundle2-input-part: "listkeys" (params: 1 mandatory) supported
   bundle2-input-part: total payload size 58
   bundle2-input-part: "listkeys" (params: 1 mandatory) supported
diff --git a/tests/test-patchbomb.t b/tests/test-patchbomb.t
--- a/tests/test-patchbomb.t
+++ b/tests/test-patchbomb.t
@@ -352,14 +352,14 @@  test bundle and description:
   Content-Disposition: attachment; filename="bundle.hg"
   Content-Transfer-Encoding: base64
   
-  SEcyMAAAAA5Db21wcmVzc2lvbj1CWkJaaDkxQVkmU1kIqE7KAAAKf//7vFYQWD1/4H7R09C/470I
-  Ak0E4peoSIYIgQCgGUQOcLABGY2hqoTTCYaBqaYAAACaaMAATIwAA1MIYDaQaqn6p+jRop+oJkA2
-  oNqD1PU0PUBoxqaMmjMUepoBoDT1GmQNBKmlTT1GTCNMEAYTQ0NNDI0BoMQHpAZAA8o2pkyNJHfX
-  RRbXoyxKRUlAg41B3lpmMOnr77dEpFKAvEUGEkWuC4wioiMjC2Y2a84EXhsNCFIrbXUGId07PJnS
-  ELAOIpL/gE8R8CUeXuw2NKMtkFoLPkcTSomXtgHSg1IKaCNlWwVU3CpmMYqh5gkFYJKOD4UhVVQ6
-  SiF1DpE8ghWvF1ih+fYgagfYHI96w/QsrRATpYiP7VRbINFrQy2c21mZ7M4pXXrPBypoXAIhtum7
-  aKDJCpUqMDF5dfiDChMfgH9nQ4B60Uvgb4AK9dsbSYc+O3tEyNq9g9gZeA5Je2T82GzjC4DbY4F2
-  0kdrTBwslErFshCgDzeEBwICg13oQaQawQA1WWd3F3JFOFCQCKhOyg==
+  SEcyMAAAAA5Db21wcmVzc2lvbj1CWkJaaDkxQVkmU1nYvy2xAAAJf//7vFYQXD1/4H7R09C/470I
+  Ak0E4pe4SIIIgQSgGEQOcLAA5VBKqeppMxTI0YjQNBgQMQDI0GgMhtR6I0GI2p6I0yeSEVT9MiYn
+  qjCYQwCBtARptARgBNDEwAGiDCMA40NGjQaNA0AAAAADIAAAA0BkABktCk6qObVxZ2A/33KHibLr
+  UQ4BwkgcPcmuCUAQZCztIWgR1SpBS6IqbIij4UFwhnnMkElcFTqoucIWbsBPK3l+6c+xYaVBWsJo
+  aT0OV/YAOvLrziifDQMJOMIaaYce9agtI2EwQBAq089UiRU+evFHSLRBT7Wa/D/YBaUtU5ezvtr3
+  6yrIS4Iyp9VWESdWPEi6VjRjdcEY4HvbmDIVEAEVJIUrHNIBx/MmnBBRkw8tSlCQ8ABZxf5ejgBI
+  pP5TSQPLVMYbq1qbBPmWN0LYVlAvRbP4X512kDQZ9y4TQbvoZmhe+54sRsEJ8GW3hMJjERh0NNlg
+  aB+3Cw/4u5IpwoSGxfltiA==
   --===============*==-- (glob)
 
 with a specific bundle type
diff --git a/tests/test-phases-exchange.t b/tests/test-phases-exchange.t
--- a/tests/test-phases-exchange.t
+++ b/tests/test-phases-exchange.t
@@ -765,7 +765,7 @@  Bare push with next changeset and common
   searching for changes
   1 changesets found
   uncompressed size of bundle content:
-       192 (changelog)
+       178 (changelog)
        165 (manifests)
        131  a-H
   adding changesets
diff --git a/tests/test-push-warn.t b/tests/test-push-warn.t
--- a/tests/test-push-warn.t
+++ b/tests/test-push-warn.t
@@ -150,7 +150,7 @@  Specifying a revset that evaluates to nu
   searching for changes
   2 changesets found
   uncompressed size of bundle content:
-       348 (changelog)
+       352 (changelog)
        326 (manifests)
        253  foo
   adding changesets
diff --git a/tests/test-rebase-conflicts.t b/tests/test-rebase-conflicts.t
--- a/tests/test-rebase-conflicts.t
+++ b/tests/test-rebase-conflicts.t
@@ -299,7 +299,7 @@  Check that the right ancestors is used w
   adding file changes
   adding f1.txt revisions
   added 2 changesets with 2 changes to 1 files
-  bundle2-input-part: total payload size 1713
+  bundle2-input-part: total payload size 1686
   bundle2-input-bundle: 0 parts total
   invalid branchheads cache (served): tip differs
   history modification detected - truncating revision branch cache to revision 9
diff --git a/tests/test-rebase-mq-skip.t b/tests/test-rebase-mq-skip.t
--- a/tests/test-rebase-mq-skip.t
+++ b/tests/test-rebase-mq-skip.t
@@ -70,14 +70,14 @@  already has one local mq patch
   $TESTTMP/a/.hg/patches/p0.patch (glob)
   2 changesets found
   uncompressed size of bundle content:
-       384 (changelog)
+       348 (changelog)
        324 (manifests)
        129  p0
        129  p1
   saved backup bundle to $TESTTMP/a/.hg/strip-backup/13a46ce44f60-5da6ecfb-backup.hg (glob)
   2 changesets found
   uncompressed size of bundle content:
-       439 (changelog)
+       403 (changelog)
        324 (manifests)
        129  p0
        129  p1
diff --git a/tests/test-rebase-newancestor.t b/tests/test-rebase-newancestor.t
--- a/tests/test-rebase-newancestor.t
+++ b/tests/test-rebase-newancestor.t
@@ -304,13 +304,13 @@  rebase of merge of ancestors
   rebase merging completed
   1 changesets found
   uncompressed size of bundle content:
-       213 (changelog)
+       199 (changelog)
        216 (manifests)
        182  other
   saved backup bundle to $TESTTMP/parentorder/.hg/strip-backup/4c5f12f25ebe-f46990e5-backup.hg (glob)
   1 changesets found
   uncompressed size of bundle content:
-       272 (changelog)
+       254 (changelog)
        167 (manifests)
        182  other
   adding branch